# Parse XML response and extract the attributes from it to form new fields **URL:** \ **Category:** Logstash\ **Created:** [April 16, 2019, 12:43pm UTC](https://discuss.elastic.co/t/parse-xml-response-and-extract-the-attributes-from-it-to-form-new-fields/177090 "2019-04-16T12:43:15Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![kaushik.vankayala](https://avatars.discourse-cdn.com/v4/letter/k/aca169/32.png) [@kaushik.vankayala](https://discuss.elastic.co/u/kaushik.vankayala)\ **Post date:** [April 16, 2019, 12:43pm UTC](https://discuss.elastic.co/t/parse-xml-response-and-extract-the-attributes-from-it-to-form-new-fields/177090/1 "2019-04-16T12:43:15Z")
Hi There, I am working on monitoring a specific queue in ActiveMQ server. For the same i am using the xml utility which returns the structure of the queue like below; This XML file does not appear to have any style information associated with it. The document tree is shown below. ```auto queueBrowse/Sample?view=rss&feedType=atom_1.0 queueBrowse/Sample?view=rss&feedType=rss_2.0 queueBrowse/PUBLISHEVENTS.UAT?view=rss&feedType=atom_1.0 queueBrowse/PUBLISHEVENTS.UAT?view=rss&feedType=rss_2.0 queueBrowse/Test?view=rss&feedType=atom_1.0 queueBrowse/Test?view=rss&feedType=rss_2.0 ``` Below is the conf file i am trying but unable to extract the attributes as expected; ``` input { http_poller { urls => { test => { method => get url => "http://localhost:8161/admin/xml/queues.jsp" user => "admin" password => "admin" headers => { Accept => "application/json" } } } request_timeout => 60 codec => "plain" schedule => { "every" => "30s" } } } filter { xml { source => "message" target => "parsed" } split { field => "[parsed][queue]" add_field => { queue_name => "%{[parsed][queue][@name]}" queue_size => "%{[parsed][queue][stats][@size]}" consumer_count => "%{[parsed][queue][stats][@consumerCount]}" enqueue_count => "%{[parsed][queue][stats][@enqueueCount]}" dequeue_count => "%{[parsed][queue][stats][@dequeueCount]}" } } mutate { #convert => { # "queue_size" => "integer" # "consumer_count" => "integer" # "enqueue_count" => "integer" # "dequeue_count" => "integer" #} remove_field => ["message", "host"] } } output { stdout { codec => rubydebug } } ``` Below is the console output; ``` { "@version" => "1", "enqueue_count" => "%{[parsed][queue][stats][@enqueueCount}]", "dequeue_count" => "%{[parsed][queue][stats][@dequeueCount}]", "consumer_count" => "%{[parsed][queue][stats][@consumerCount]}", "@timestamp" => 2019-04-16T12:42:23.382Z, "queue_size" => "%{[parsed][queue][stats][@size]}", "queue_name" => "%{[parsed][queue][@name]}" } { "@version" => "1", "enqueue_count" => "%{[parsed][queue][stats][@enqueueCount}]", "dequeue_count" => "%{[parsed][queue][stats][@dequeueCount}]", "consumer_count" => "%{[parsed][queue][stats][@consumerCount]}", "@timestamp" => 2019-04-16T12:42:23.382Z, "queue_size" => "%{[parsed][queue][stats][@size]}", "queue_name" => "%{[parsed][queue][@name]}" } { "@version" => "1", "enqueue_count" => "%{[parsed][queue][stats][@enqueueCount}]", "dequeue_count" => "%{[parsed][queue][stats][@dequeueCount}]", "consumer_count" => "%{[parsed][queue][stats][@consumerCount]}", "@timestamp" => 2019-04-16T12:42:23.382Z, "queue_size" => "%{[parsed][queue][stats][@size]}", "queue_name" => "%{[parsed][queue][@name]}" } ``` Please kindly help how we can extract the attributes! Regards Kaushik ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [April 16, 2019, 1:07pm UTC](https://discuss.elastic.co/t/parse-xml-response-and-extract-the-attributes-from-it-to-form-new-fields/177090/2 "2019-04-16T13:07:07Z")
> [@kaushik.vankayala](#): > > consumer\_count =\> "%{[parsed][queue][stats][@consumerCount}]" Close, but not quite right. Remove the trailing ]. Remove the @. And stats is an array, so you need to provide an index. ``` consumer_count => "%{[parsed][queue][stats][0][consumerCount}" ``` Same changes for the others, except queue\_name, which does not need an array index. ---
**Author:** ![kaushik.vankayala](https://avatars.discourse-cdn.com/v4/letter/k/aca169/32.png) [@kaushik.vankayala](https://discuss.elastic.co/u/kaushik.vankayala)\ **Post date:** [April 17, 2019, 3:44am UTC](https://discuss.elastic.co/t/parse-xml-response-and-extract-the-attributes-from-it-to-form-new-fields/177090/3 "2019-04-17T03:44:56Z")
@Badger, thank for your time in replying to this. If i have to only get the stats of a specific queue, like say in the above XML i want to extract the data of only the queue name "PUBLISHEVENTS.UAT" any idea how do i go about it? Note: The index may not be same all the time. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [April 17, 2019, 1:04pm UTC](https://discuss.elastic.co/t/parse-xml-response-and-extract-the-attributes-from-it-to-form-new-fields/177090/4 "2019-04-17T13:04:48Z")
You could extract a specific element from the array using ruby ``` ruby { code => ' a = event.get("[@metadata][theXML][queue]") a.each_index { |x| if a[x]["name"] == "PUBLISHEVENTS.UAT" event.set("queueStats", a[x]) end } ' } ``` ---
**Author:** ![kaushik.vankayala](https://avatars.discourse-cdn.com/v4/letter/k/aca169/32.png) [@kaushik.vankayala](https://discuss.elastic.co/u/kaushik.vankayala)\ **Post date:** [April 22, 2019, 12:59pm UTC](https://discuss.elastic.co/t/parse-xml-response-and-extract-the-attributes-from-it-to-form-new-fields/177090/5 "2019-04-22T12:59:16Z")
Hi @Badger, I am yet to try your provided solution. However, i tried to achieve my requirement using drop plugin with an if condition as below; ``` if "PUBLISHEVENTS.UAT" not in [queue_name] { drop {} } ``` But i now need to add a files with the size of the > [@kaushik.vankayala](#): > > \ queues data like in the above xml data it is 3 Could you drop some insight over? I actually have a python script which outputs a json by parsing the xml received in the above poller. Any insights? Regards Kaushik. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [April 24, 2019, 2:24pm UTC](https://discuss.elastic.co/t/parse-xml-response-and-extract-the-attributes-from-it-to-form-new-fields/177090/6 "2019-04-24T14:24:58Z")
Insert this between the xml and split filters ``` ruby { code => 'event.set("numOfQueues", event.get("[parsed][queue]").length)' } ``` ---
**Author:** ![kaushik.vankayala](https://avatars.discourse-cdn.com/v4/letter/k/aca169/32.png) [@kaushik.vankayala](https://discuss.elastic.co/u/kaushik.vankayala)\ **Post date:** [April 25, 2019, 5:13am UTC](https://discuss.elastic.co/t/parse-xml-response-and-extract-the-attributes-from-it-to-form-new-fields/177090/7 "2019-04-25T05:13:23Z")
Works like a charm. 😉 I have checked the ruby filter plugin documentation but did not find it much helpful. Do you have any good references for ruby scripting? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 23, 2019, 5:13am UTC](https://discuss.elastic.co/t/parse-xml-response-and-extract-the-attributes-from-it-to-form-new-fields/177090/8 "2019-05-23T05:13:34Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.