Parsing XML file

[Makra (Makara)]

Hi
I am trying to parse this XML file to fetch all the <item name="Software Application"> </item> tag and the property inside them under <category name="Software"> </category> . I also need to count no of <item name="Software Application"> tag that are there. Is it possible that each item tag must be a doc thereby i can differentiate the properties under which item/category tag it falls. I have written the logstash configuration file, but the log says.

[2017-06-09T14:04:04,247][DEBUG][logstash.inputs.file ] _globbed_files: /home/xmldata/test.xml: glob is:
[2017-06-09T14:04:05,175][DEBUG][logstash.pipeline ] Pushing flush onto pipeline
[2017-06-09T14:04:10,175][DEBUG][logstash.pipeline ] Pushing flush onto pipeline
[2017-06-09T14:04:15,176][DEBUG][logstash.pipeline ] Pushing flush onto pipeline
[2017-06-09T14:04:19,253][DEBUG][logstash.inputs.file ] _globbed_files: /home/xmldata/test.xml: glob is:
[2017-06-09T14:04:20,176][DEBUG][logstash.pipeline ] Pushing flush onto pipeline
[2017-06-09T14:04:25,175][DEBUG][logstash.pipeline ] Pushing flush onto pipeline

The configuration file is -

input {
 file {
         path => "/home/xmldata/test.xml"
         start_position => beginning
         sincedb_path => "/home/xmldata/sincedb/test"
                codec => multiline
                 {
                   pattern => "<item name='Software Application'>|</item>"
                   negate => true
                   what => "previous"
                   auto_flush_interval => 1
                  }
      }
}
filter {
  if [message] == "<category/>" or [message] == "</category>" {
    drop {}
  }
  xml {
    source => "message"
    target => "message.parsed"
    xpath => [ "/item/message/text()", itemmessage ]
    force_array => false
  }
}

output {
        stdout { 
                codec => rubydebug 
               }
                elasticsearch {
                                  index => "logstash-xml-%{+dd.MM.YYYY}"
                                  hosts => ["localhost:9200"]
                                  document_id => "%{[id]}"
                                  document_type => "xmlfiles"
                }
} 

Is it the right way to parse an XML file in logstash ?
Regards

[Makra (Makara)]

Hi
I have entirely modified the logstash conf but now, i am getting ->
Only String and Array types are splittable. field:[parsed][Software Application] is of type = NilClass

The modified configuration file is->

input {
file {
path => "/home/xmldata/test.xml"
start_position => beginning
sincedb_path => "/dev/null"
ignore_older => 0
#codec => "plain"
}
}

filter {
xml {
source => "message"
target => "parsed"
}
split {
# type => "string"
field => "[parsed][Software Application]"

            add_field => {
                            Name => "%{[parsed][Software Application][name]}"
            }
          }

}

Does the field [parsed][Software Application] type is not string ? I tried to convert it using type => "string" but that does not seems to work.

[magnusbaeck (Magnus Bäck)]

Please show an example event that Logstash has processed, e.g. by adding a stdout { codec => rubydebug } output (copy/paste from Kibana's JSON tab is also okay).