How to add_field and covert field type while parsing nested xml

(Saket Kumar) #1
<?root> <?fields1> <?fields1.1> <?fields1.1.1> <?fielda>...<?/fielda> <?fieldb>...<?/fieldb> ..... Requirement is something like i would like add field with custom name add_field => { custom_field_name => "%{[root][fields1][fields1.1][fielda]}" custom_field_name => "%{[root][fields1][fields1.1][fielda]}" } Also, mutate {convert => ["root.fields1.fields1.1.fielda", "integer"]} I tried both the way but didnt get success. Is it correct way to config. Thanks

(Saket Kumar) #2

Any help is much appreciated!

(Magnus Bäck) #3

Are you able to parse the XML correctly? What do your messages look like right now?

(Saket Kumar) #4

Yes now by replacing Logstash from 1.5.0 rc3 to 1.5.0 the exception is eradicated. Found on forum that error is fixed in higher version.

message looks like....
{"message":"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n\n200\nOk\n1\n\n150603_VD_9VJ\n1009017\n979560\n17\n61\n\n","@version":"1","@timestamp":"2015-06-08T08:38:50.056Z","host":"mac109","path":"/opt/Log/new7.xml","tags":["multiline","_xmlparsefailure"],"data":{"statusCode":["200"],"statusText":["Ok"],"requestId":["1"],"data":[{"testId":["150603_VD_9VJ"],"bytesIn":["1009017"],"bytesInDoc":["979560"],"connections":["17"],"requests":["61"]}]}}

(Magnus Bäck) #5

I don't know exactly what output you want, but this extracts the testId and bytesIn fields into top-level fields and converts the latter to an integer:

mutate {
  add_field => {
    "test_id" => "%{[data][0][testId][0]}"
    "bytes_in" => "%{[data][0][bytesIn][0]}"
mutate {
  convert => ["bytes_in", "integer"]

(Saket Kumar) #6

Thanks magnus! i was expecting same to understand how would I be refering fieldarray...
thanks for your help...

(system) #7