Upload xml file into elasticsearch

Elastic Stack Elasticsearch
1

Hi everyone,
i want that logstash can groupe the data of each section of my xml file, what i should to modify on remove in my config file. for example i want my result like that :

 "_index": "logstash-xml",
    "_type": "xmlfiles",
    "_source": {
    "key1": 001
    "id": 001
    "date": 01-01-2016

    "_index": "logstash-xml",
    "_type": "xmlfiles",
    "_source": {
    "key1": 002
    "id": 002
    "date": 02-02-2016

This my config file :

 input {
 file {
 path => "D:\server\xmldata.xml"
 start_position => beginning
 codec => multiline
 {
  pattern => "^<\?xmldata .*\>"
  negate => true
  what => "previous"
 }
}
}

filter {
xml {
store_xml => false
source => "message"
xpath =>
[
"/xmldata/head1/id/text()", "id",
"/xmldata/head1/date/text()", "date",
"/xmldata/head1/key1/text()", "key1"
]
}

date {
match => [ "date" , "dd-MM-yyyy HH:mm:ss" ]
timezone => "Europe/Amsterdam"
}

}

output {
stdout { codec => rubydebug }
elasticsearch {
index => "logstash-xml"
hosts => ["localhost:9200"]
document_id => "%{[id]}"
document_type => "xmlfiles"
}
}

This is my xml file :

 <xmldata>
 <head1>
  <key1>Value1</key1>
  <id>0001</id>
  <date>01-01-2016 09:00:00</date>
 </head1>
 <head1>
  <key1>Value2</key1>
  <id>0002</id>
  <date>02-02-2016 02:00:00</date>
 </head1>
</xmldata>
2

Hello,

please find the sample example to load xml file here.

Hope this helps you

3

Thanks but i got the same old resultat unfortunately .... ;(

4

Hello,

can you show me what changes you have applied to the input code. so that it would be easy to provide solution for you.

Regards

5

input {
file {
path => "D:\server\xmldata.xml"
start_position => beginning
codec => multiline
{
pattern => "^<?xmldata .*>"
negate => true
what => "previous"
}
}
}

6

change path as shown below, please follow as shown in example properly

path => "D:/server/xmldata.xml"

7

i got the same result :

image
image.png1351×710 33.9 KB

8

it s better if i can groupe data of each section like in this image

image
image.png707×241 6.9 KB

9

Are you looking for something like this?

input {
  generator {
    lines => ['<xmldata>
 <head1>
  <key1>Value1</key1>
  <id>0001</id>
  <date>01-01-2016 09:00:00</date>
 </head1>
 <head1>
  <key1>Value2</key1>
  <id>0002</id>
  <date>02-02-2016 02:00:00</date>
 </head1>
</xmldata>']
    count => 1
  } 
} 

filter {
  xml {
    source => "message"
    target => "data"
  }

  split {
    field => "[data][head1]"
  }

  mutate {
    add_field => { "key1" => "%{[data][head1][key1][0]}"
                   "id" => "%{[data][head1][id][0]}"
                   "date" => "%{[data][head1][date][0]}"
                 }
  }

  mutate {
    remove_field => ["data", "message"]
  }
}

output {
  stdout { codec => rubydebug }
}
1 Like
10

Thanks a lot its work like i want

11

and what i should to do if i dont want to paste all xml file in the generator for example replace this code :
image

with this new code ;

image

but it doesnt work :frowning: @Christian_Dahlqvist @balumurari1

12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.