Need to parse xml by logstash and create an event only if a field value is non zero

Elastic Stack Logstash
1

Hi Team ,

I have few Jenkins build xml files generated when jenkins jobs are triggered. I have to read few tags from the xml and create new fields from them by parsing it through logstash.

The xml has one tag <duration> which has a value 0 in the start but it gets updated to actual value once jenkins job is completed. My logstash is reading the xml and sending value 0 in event but not reading the updated tag once xml is updated .

is there a way my xml is only read by logstash if my xml tag value in <duration> is non zero. and if zero to skip the parsing of xml.

below is my logstash conf:

logstash.conf: |
    input {
      file {
        path => "/var/jenkins_home/jobs/**/branches/*/builds/*/build.xml"
        start_position => "beginning"
        sincedb_path => "/dev/null"
        type => "xml"
        codec => multiline {
           pattern => '^[A-Z]{1}[a-z]{2} {1,2}[0-9]{1,2},[0-9]{4} {1,2}[0-9]{1,2}:[0-9]{2}:[0-9]{2}'
           negate => true
           what => previous
           max_lines => 10000000000
           auto_flush_interval => 60
        }
      }
    }
    filter {
      xml {
        source => "message"
        store_xml => false
        xpath => [
            "/flow-build/startTime/text()", "startTime",
            "/flow-build/duration/text()", "duration",
            "/flow-build/execution/result/text()", "result"
        ]
        remove_field => [ "message" ]
      }
    }
   output {
      elasticsearch { hosts => [ "https://elastic:443/elasticsearch" ] index => "elktest-%{+YYYY.MM.dd}" }
      stdout { codec => rubydebug }
    }
2

Hi,

First, to make logstash read the file when he is edited, you need to set the sincedb_path properly. I let you read the documentation.

Next, to not send the values to Elasticsearch if the field duration is equals to zero, i recommend you to use one conditionnal. You have an example here and the documentation of the conditionnal is here.

Cad.