Logstash Filter for XML file

Elastic Stack Logstash
1

Hi,

I am looking to parse multiple XML files and can't find a suitable filter in Logstash for the same. I've around 50 XML files in different sub folders but all should have same structure as it is an output of pnp4nagios. I am looking to parse and send the below key values to elasticsearch.

<NAGIOS_AUTH_HOSTNAME>remotehost1</NAGIOS_AUTH_HOSTNAME>
<NAGIOS_AUTH_SERVICEDESC>Zombie Processes</NAGIOS_AUTH_SERVICEDESC>
<NAGIOS_PERFDATA>procs=0;5;10;0; </NAGIOS_PERFDATA>
<NAGIOS_SERVICEPERFDATA>procs=0;5;10;0;</NAGIOS_SERVICEPERFDATA>
<NAGIOS_SERVICESTATE>OK</NAGIOS_SERVICESTATE>

Here is my sample XML file.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?> check_nrpe /usr/local/pnp4nagios/var/perfdata/remotehost1/Zombie_Processes.rrd SINGLE 8460 0 1 procs procs 0 5 10 0 0 successful updated remotehost1 Zombie Processes check_nrpe!check_zombie_procs SERVICEPERFDATA remotehost1 Zombie Processes remotehost1 UP HARD procs=0;5;10;0; /usr/local/pnp4nagios/var/perfdata/remotehost1/Zombie_Processes.rrd check_nrpe!check_zombie_procs Zombie_Processes procs=0;5;10;0; OK HARD 1518688537 /usr/local/pnp4nagios/var/perfdata/remotehost1/Zombie_Processes.xml 4
2

Here is my sample XML file.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?> check_nrpe /usr/local/pnp4nagios/var/perfdata/remotehost1/Zombie_Processes.rrd SINGLE 8460 0 1 procs procs 0 5 10 0 0 successful updated remotehost1 Zombie Processes check_nrpe!check_zombie_procs SERVICEPERFDATA remotehost1 Zombie Processes remotehost1 UP HARD procs=0;5;10;0; /usr/local/pnp4nagios/var/perfdata/remotehost1/Zombie_Processes.rrd check_nrpe!check_zombie_procs Zombie_Processes procs=0;5;10;0; OK HARD 1518688537 /usr/local/pnp4nagios/var/perfdata/remotehost1/Zombie_Processes.xml 4

This isn't XML. Try formatting it as preformatted text using the </> toolbar button so the forum software doesn't strip the XML tags.

3 
<?xml version="1.0" encoding="UTF-8"?>
<NAGIOS>
   <DATASOURCE>
      <TEMPLATE>check_nrpe</TEMPLATE>
      <RRDFILE>/usr/local/pnp4nagios/var/perfdata/remotehost1/Zombie_Processes.rrd</RRDFILE>
      <RRD_STORAGE_TYPE>SINGLE</RRD_STORAGE_TYPE>
      <RRD_HEARTBEAT>8460</RRD_HEARTBEAT>
      <IS_MULTI>0</IS_MULTI>
      <DS>1</DS>
      <NAME>procs</NAME>
      <LABEL>procs</LABEL>
      <UNIT />
      <ACT>0</ACT>
      <WARN>5</WARN>
      <WARN_MIN />
      <WARN_MAX />
      <WARN_RANGE_TYPE />
      <CRIT>10</CRIT>
      <CRIT_MIN />
      <CRIT_MAX />
      <CRIT_RANGE_TYPE />
      <MIN>0</MIN>
      <MAX />
   </DATASOURCE>
   <RRD>
      <RC>0</RC>
      <TXT>successful updated</TXT>
   </RRD>
   <NAGIOS_AUTH_HOSTNAME>remotehost1</NAGIOS_AUTH_HOSTNAME>
   <NAGIOS_AUTH_SERVICEDESC>Zombie Processes</NAGIOS_AUTH_SERVICEDESC>
   <NAGIOS_CHECK_COMMAND>check_nrpe!check_zombie_procs</NAGIOS_CHECK_COMMAND>
   <NAGIOS_DATATYPE>SERVICEPERFDATA</NAGIOS_DATATYPE>
   <NAGIOS_DISP_HOSTNAME>remotehost1</NAGIOS_DISP_HOSTNAME>
   <NAGIOS_DISP_SERVICEDESC>Zombie Processes</NAGIOS_DISP_SERVICEDESC>
   <NAGIOS_HOSTNAME>remotehost1</NAGIOS_HOSTNAME>
   <NAGIOS_HOSTSTATE>UP</NAGIOS_HOSTSTATE>
   <NAGIOS_HOSTSTATETYPE>HARD</NAGIOS_HOSTSTATETYPE>
   <NAGIOS_MULTI_PARENT />
   <NAGIOS_PERFDATA>procs=0;5;10;0;</NAGIOS_PERFDATA>
   <NAGIOS_RRDFILE>/usr/local/pnp4nagios/var/perfdata/remotehost1/Zombie_Processes.rrd</NAGIOS_RRDFILE>
   <NAGIOS_SERVICECHECKCOMMAND>check_nrpe!check_zombie_procs</NAGIOS_SERVICECHECKCOMMAND>
   <NAGIOS_SERVICEDESC>Zombie_Processes</NAGIOS_SERVICEDESC>
   <NAGIOS_SERVICEPERFDATA>procs=0;5;10;0;</NAGIOS_SERVICEPERFDATA>
   <NAGIOS_SERVICESTATE>OK</NAGIOS_SERVICESTATE>
   <NAGIOS_SERVICESTATETYPE>HARD</NAGIOS_SERVICESTATETYPE>
   <NAGIOS_TIMET>1518688537</NAGIOS_TIMET>
   <NAGIOS_XMLFILE>/usr/local/pnp4nagios/var/perfdata/remotehost1/Zombie_Processes.xml</NAGIOS_XMLFILE>
   <XML>
      <VERSION>4</VERSION>
   </XML>
</NAGIOS>`Preformatted text`
4

Just use an xml filter. Its xpath option makes it easy to extract the contents of certain XML tags into certain fields.

5

Thank you very much for you advise. I am fairely new to this topic and not sure how to use an xml filter. Is it possible to provide me an example.

6

There are lots of examples in post threads, e.g. here: Need a complete XML Filter example

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.