Filtering and Processing XML Logs in Logstash for JSON Conversion

Elastic Stack Logstash
1

Hi All,

I want to use only a few parameterized XML tags to convert into JSON and remove all other parameterized tags before processing.

My XML log looks like this:

<a>
    <b direction="outgoing">
      <c id="0" value="1"/>
      <c id="3" value="2"/>
      <c id="4" value="3"/>
      <c id="11" value="4"/>
      <c id="12" value="5"/>
    </b>
</a>

I only need the tags with id=0 and id=3 and want to remove the remaining tags before processing the XML log for JSON migration.

This is how my XML logs should be processed:

<a>
    <b direction="outgoing">
      <c id="0" value="1"/>
      <c id="3" value="2"/>
    </b>
</a>
2

You could try something like

    xml {
        source => "message"
        store_xml => true
        target => "theXML"
        force_array => false
    }
    ruby {
        code => '
            xml = event.get("theXML")
            c = xml["b"]["c"]
            if c.is_a? Array
                keep = ["0", "3"]
                c.each_with_index { |x, i|
                    unless keep.include? x[id]
                        event.remove("[theXML][b][c][#{i}]")
                    end
                }
            end
        '
    }