Integer comparison not working

Elastic Stack Logstash
1

I'm trying to conditionally set some fields in my pipeline:

filter {
  if [log][file][path] == "/var/log/e2guardian/access.log" {
    grok {
      match => { "message" => "(?<e2_timestamp>\d\d\d\d\.\d\d\.\d\d \d\d:\d\d:\d\d)\t%{NOTSPACE:user.name}\t%{IP:client.address}\t%{NOTSPACE:url.full}\t%{NOTSPACE:event.action} ?(hw\d+: )?(?<why>[^\t]+)?\t%{WORD:http.request.method}\t%{NUMBER:http.request.bytes}\t\d+\t[^\t]+\t[^\t]+\t%{NUMBER:http.response.status_code}\t[^\t]+\t%{IP:client.ip}" }
    }
    if "_grokparsefailure" not in [tags] {
      mutate {
        remove_field => [ "message" ]
        add_field => { "[event][category]" => [ "network", "web" ] }
        add_field => { "[event][kind]" => "event" }
        convert => { "[http][response][status_code]" => "integer" }
      }
      if [http][response][status_code] and [http][response][status_code] >= 200 and [http][response][status_code] < 400 {
        mutate {
          add_field => { "[event][type]" => [ "access", "connection", "allowed" ] }
          add_field => { "[event][outcome]" => "success" }
        }
      } else {
        mutate {
          add_field => { "[event][type]" => [ "access", "connection", "denied" ] }
          add_field => { "[event][outcome]" => "failure" }
        }
      }
    }
    date {
      match => [ "e2_timestamp", "yyyy.MM.dd HH:mm:ss" ]
      timezone => "%{[event][timezone]}"
      remove_field => [ "e2_timestamp" ]
    }
  }
}

But entries with http.response.status code are still ending up with "denied" and "failure":

    "http.response.status_code": "200",
    "event": {
      "category": [
        "network",
        "web"
      ],
      "type": [
        "access",
        "connection",
        "denied"
      ],
      "outcome": "failure",
      "timezone": "-06:00",
      "kind": "event"
    },
2

In your grok filter you created a field http.response.status_code (one field with dots in the name), not [http][response][status_code] (nested field). So the field you are trying to convert to integer and use in your condition simply does not exist.

3

Egads, somehow I believed that the two notations were equivalent. Is there any difference in how the data is stored in elasticsearch? Is there any advantage of working with one form or the other in logstash? Thank you!

4

Here's a quite recent discussion on that topic:

1 Like
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.