XML Filter Share value between events

Elastic Stack Logstash
1

Hi all,
I need to ingest XML files which are the output of a Nessus scanner.
The XML has this structure

?xml version="1.0" ?>
<NessusClientData_v2>
<Report name="host1" xmlns:cm="http://www.nessus.org/cm">
<ReportHost name="Host1">
<HostProperties>
....
</HostProperties>
<ReportItem port="80" svc_name="www" protocol="tcp" severity="2" pluginID="45411" pluginName="SSL" pluginFamily="General">
	<tag1>value1</tag1>
	<tag2>value2</tag2>
	<tag3>value3</tag3>
</ReportItem>

<ReportItem port="8081" svc_name="www" protocol="tcp" severity="2" pluginID="45411" pluginName="SSL" pluginFamily="General">
	<tag1>value1</tag1>
	<tag2>value2</tag2>
	<tag3>value3</tag3>
</ReportItem>

<ReportItem port="8083" svc_name="www" protocol="tcp" severity="2" pluginID="45411" pluginName="SSL" pluginFamily="General">
	<tag1>value1</tag1>
	<tag2>value2</tag2>
	<tag3>value3</tag3>
</ReportItem>
<ReportHost name="Host2">
<HostProperties>
....
</HostProperties>
<ReportItem port="80" svc_name="www" protocol="tcp" severity="2" pluginID="45411" pluginName="SSL" pluginFamily="General">
	<tag1>value1</tag1>
	<tag2>value2</tag2>
	<tag3>value3</tag3>
</ReportItem>

<ReportItem port="8081" svc_name="www" protocol="tcp" severity="2" pluginID="45411" pluginName="SSL" pluginFamily="General">
	<tag1>value1</tag1>
	<tag2>value2</tag2>
	<tag3>value3</tag3>
</ReportItem>

<ReportItem port="8083" svc_name="www" protocol="tcp" severity="2" pluginID="45411" pluginName="SSL" pluginFamily="General">
	<tag1>value1</tag1>
	<tag2>value2</tag2>
	<tag3>value3</tag3>
</ReportItem>
</Report name>
</NessusClientData_v2>

I need to have one document per ReportItem but I want to add each document a new field containing the host information.
Is it possible to share the name of the tah ReportHost in the preceding ReportItems events?
I have tried something like the configuration below, with no luck
How can I carry the hostname between events until a new ReportHost appears?
Thank you!
Regards
Ana

input {
  file {
    path => "/data/nessus/nessus.xml"
    sincedb_path => "/dev/null"
    start_position => "beginning"
    codec => multiline {
      pattern => "^<ReportItem |^<ReportHost"
      auto_flush_interval => 1
      negate => "true"
      what => "previous"
      max_lines => 1000000000
     max_bytes => "50 MiB"

    }
      tags => "xml_test"
      type => "xml_test"
  }
}

filter {
  ##interpret the message as XML
        if [message] =~ "^<ReportHost*" {
                xml{
                        source => "message"
                        store_xml => "false"
                        force_array => "false"
                        xpath => ["ReportHost/@name", name]
                }
                ruby {
                        init => "@asset = event.get('name')"
                }
        }

        if [message] =~ "^<ReportItem*" {
                xml{
                        source => "message"
                        store_xml => "false"
                        force_array => "false"
                        xpath => ["ReportItem/@tag1", plugin_name]
                        xpath => ["ReportItem/@tag2", plugin_id]
                        xpath => ["ReportItem/@tag3", risk_score]
                }

                ruby {
                code => "
                          event.set('asset', @asset')
                "
                }
        }
}
2

If your XML is actually valid (e.g. it does not contain constructs such as " </Report name>") then if you ingest the entire file as a single event you can use something like

    xml { source => "message" target => theXML }
    split { field => "[theXML][Report][0][ReportHost]" }
    split { field => "[theXML][Report][0][ReportHost][ReportItem]" }

then you can mutate+rename to give the fields more useable names.

3

Thank you @Badger.
The XML is ok (I put a fragment of the XML in the message and I forgot to remove the /Report name tag).
It took me a while to understand the solution, but finally It worked
Thanks a lot!
Regards
Ana

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.