Need to parse xml by logstash and create an event only if a field value is non zero

Hi Team ,

I have few Jenkins build xml files generated when jenkins jobs are triggered. I have to read few tags from the xml and create new fields from them by parsing it through logstash.

The xml has one tag <duration> which has a value 0 in the start but it gets updated to actual value once jenkins job is completed. My logstash is reading the xml and sending value 0 in event but not reading the updated tag once xml is updated .

is there a way my xml is only read by logstash if my xml tag value in <duration> is non zero. and if zero to skip the parsing of xml.

below is my logstash conf:

logstash.conf: |
    input {
      file {
        path => "/var/jenkins_home/jobs/**/branches/*/builds/*/build.xml"
        start_position => "beginning"
        sincedb_path => "/dev/null"
        type => "xml"
        codec => multiline {
           pattern => '^[A-Z]{1}[a-z]{2} {1,2}[0-9]{1,2},[0-9]{4} {1,2}[0-9]{1,2}:[0-9]{2}:[0-9]{2}'
           negate => true
           what => previous
           max_lines => 10000000000
           auto_flush_interval => 60
        }
      }
    }
    filter {
      xml {
        source => "message"
        store_xml => false
        xpath => [
            "/flow-build/startTime/text()", "startTime",
            "/flow-build/duration/text()", "duration",
            "/flow-build/execution/result/text()", "result"
        ]
        remove_field => [ "message" ]
      }
    }
   output {
      elasticsearch { hosts => [ "https://elastic:443/elasticsearch" ] index => "elktest-%{+YYYY.MM.dd}" }
      stdout { codec => rubydebug }
    }

Hi,

First, to make logstash read the file when he is edited, you need to set the sincedb_path properly. I let you read the documentation.

Next, to not send the values to Elasticsearch if the field duration is equals to zero, i recommend you to use one conditionnal. You have an example here and the documentation of the conditionnal is here.

Cad.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.