XML Filter fail

Elastic Stack Logstash
1

Hi,

I'm looking to extract an xml file into usable components. On all my configs to date, the inputs have usually been syslogs or files that allow key/value separation making things easier to separate.

I have a file with multiple lines that look like this:

<Event><System><Provider Name="NetApp-Security-Auditing" Guid="{3CB2A168-FE19-4A4E-BDAD-DCF422F13473}"/><EventID>4656</EventID><EventName>Open Object</EventName><Version>101.3</Version><Source>CIFS</Source><Level>0</Level><Opcode>0</Opcode><Keywords>0x8020000000000000</Keywords><Result>Audit Success</Result><TimeCreated SystemTime="2017-12-15T10:34:51.979061000Z"/><Correlation/><Channel>Security</Channel><Computer>main-server/server-data</Computer><ComputerUUID>cf380853-6606-11e6-9638-00a098a5e1db/2fe0edc3-723f-11e7-ab83-00a098a627d4</ComputerUUID><Security/></System><EventData><Data Name="SubjectIP" IPVersion="4">172.16.0.22</Data><Data Name="SubjectUnix" Uid="65534" Gid="65534" Local="false"></Data><Data Name="SubjectUserSid">S-1-5-21-1997283580-3459341067-486214353-122727</Data><Data Name="SubjectUserIsLocal">false</Data><Data Name="SubjectDomainName">my_domain</Data><Data Name="SubjectUserName">firstname.lastname</Data><Data Name="ObjectServer">Security</Data><Data Name="ObjectType">Directory</Data><Data Name="HandleID">000000000004cc;00;00000061;2a5f8706</Data><Data Name="ObjectName">(server_data);/server_data</Data><Data Name="AccessList">%%4416 %%4423 </Data><Data Name="AccessMask">81</Data><Data Name="DesiredAccess">Read Data; List Directory; Read Attributes; </Data><Data Name="Attributes"></Data></EventData></Event>

There are many of these lines in the file. The example above is from a single line.

With kv, I could separate them and drop the ones I don't need. I'm not sure how to achieve this with an xml file.

This is as far as I've gotten. I've every expectation that I'm way off...

input {
    file {
        type => "netapp"
    	path => "C:\audit.xml"
		start_position => beginning
    	sincedb_path => "/dev/null"
		codec => multiline {
		pattern => "<Event>" 
		}
	}
}

    filter {
      xml	{
    	source => "message"
        target => "xml_content"
        xpath => [ "<Event>", "event" ]
      }
    }

output {
if [type] == "netapp" {
elasticsearch {
index => "netapp-%{+YYYY.MM.dd}"
	hosts => ["10.20.40.50:9200"]
		}
  }
}

Despite the config passing the config check, it causes errors in the Logstash logs and refuses to import. I'm essentially looking for a way to take each of the components, drop the ones I don't care about such as the Guid and then send the rest on to Elasticsearch.

Can anyone provide some pointers/explanations. I can't find any good examples of working xml filters to base mine from.

Thanks for your help.

2

First of all, is your multiline codec working as expected? Secondly, your XPath expression is... not XPath.

Save yourself some time while debugging by using a stdout { codec => rubydebug } output and disabling the elasticsearch output until you're ready to move on.

Despite the config passing the config check, it causes errors in the Logstash logs and refuses to import.

If you want help getting rid of errors you have to show us what those errors look like.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.