How to split and filtring xml in LogStash

Elastic Stack Logstash
1

I have the following logstash conf file:

input {
     file {
          path => "C:/logstash/data/test.xml"
          start_position => beginning
          codec => multiline
          {
           pattern => "^<\?providerConfigResponse .*\>"
           negate => true
           what => "previous"
          }
     }


    filter {
          xml {
           store_xml => false
           source => "message"
           xpath =>
           [
            "/providerConfigResponse//svgconfig", "svConfig",
            "/providerConfigResponse//svgconfig//function","function"

           ]
        }
    }

    output {
    elasticsearch {
        index => "xmltest"
        document_type => "xmlfiles"
        hosts => "localhost:9200"
    }

    stdout { codec => json }

and my file xml :

<providerConfigResponse>
<svgconfig>
    <name>SVG1</name>
    <function>
        test 1
    </function> 
    <function>
        test 2
    </function> 
</svgconfig>
<svgconfig>
    <name>SVG2</name>
    <function>
    test 3
    </function> 
</svgconfig>
<svgconfig>
    <name>SVG3</name>
    <function>
    test 4
    </function> 
    <function>
    test 5
    </function> 
</svgconfig>
</providerConfigResponse>

and i want to have ouput json liks this :

 {

    "svgconfig" : [function: test 1, function: test 2],
    "svgconfig" : test 3,
   " svgconfig" : [function: test 4, function: test 5]

    }

But with my config i don't have what i want

I use logstash v5.4.0 and elasticsearch v5.4.0

Any solution ?

thanks

2 
 {
    "svgconfig" : [function: test 1, function: test 2],
    "svgconfig" : test 3,
    "svgconfig" : [function: test 4, function: test 5]
  }

This isn't really possible in JSON as it is a object and so the keys must be unique (in your example you have 3 svgconfig keys). also you want that, if there's only one function block, that there is no array and just the value? that complicates things :confused:

3

Ok thank's. I want get list of svgconfig and for each item i get list of function to visualise it in Kibanna.

How i can do it ?

4

what kind of vizualization would you like? how many
blocks does each file have?

Do you just want to plot the values of the functions? do they have to be grouped by svgconfig block? if so, how do you want them grouped?

5

i want group the value of function by svgconfig .
for example the svgconfig with name 'SVG1' i have 3 functions (test1, test2 and test 3).
i edit my file xml, now is copmlet

6

ok so we have:

SVG1
   test 1
   test 2
SVG2
   test 3
SVG3
   test 4
   test 5

What kind of vizualization are you thinking about? if it was a line chart, what would be the x-axis and the y-axis?

7

Vertical Bar. for y-axis : count and x-axis :

  1. aggregation term by [svgconfig]/[name]
  2. Split Series : aggregation term by [function]
8

I'm not very well versed with elasticsearch, but I think that, for that purpose, you need to send each function in a separate event, so

SVG1, test 1
SVG1, test 2
SVG2, test 3
SVG3, test 4
SVG3, test 5

I'm not sure how to do this in xpath as I'm not an expert, maybe chain two xml filters, one that breaks each svgconfig, then one that breaks each function

9

thank's, I will test it, and I come back to you :slight_smile: