If conditional " in " doesn't work, but " == " does? I need the "in" conditional to work!

Elastic Stack Logstash
1

In my logstash .conf, filter section when I use "==" it works, but when I replace with "in" it doesn't. Further down in the script, I do use "in" with multiple values, and none of them work either. Am I missing something?

"==" succeeds

filter {
  if [app_id.keyword] == ["LoggingService"] {
        grok {

"in" fails

filter {
  if [app_id.keyword] in ["LoggingService"] {
        grok {

RHEL 7, Logstash 6.2.0 then updated to 6.2.3, still doesn't work.

2

IIRC, The right-hand-side of an in clause gets parsed as a field reference, not as a literal array as you're attempting.

Unfortunately, the grammar is ambiguous, and field references have a higher precedence than literal arrays, so there's not much we can do to fix it without breaking the situation when people do intentionally have a field reference there.

While it is certainly more verbose, using equality along-side or clauses is going to be your best bet:

if [field] == "a" or [field] == "b" {
  # ...
}
3

Thank you for explaining.

4

I made the change, but it only worked with the [] around my string, ["LoggingService"].

[app_id.keyword] == ["LoggingService"]
5

I just retried and this works:

  if [app_id.keyword] == ["LoggingService"] {

This doesn't

  if [app_id.keyword] == "LoggingService" {

Here's the whole file:

input {

  beats {
    port => 5044
#    congestion_threshold => "999999999"
  }
}

filter {
  if [app_id.keyword] == ["LoggingService"] {
        grok {
          patterns_dir => ["/etc/logstash/conf.d/patterns"]
          match => ["message", "%{GREEDYDATA:Message}Timestamp : %{TIMESTAMP_ISO8601:logtime}"]
        }
        date {
          match => ["logtime", "ISO8601", "yyyy-MM-dd HH:mm:ss.SSSS", "yyyy-MM-dd HH:mm:ss,SSS", "yyyy-MM-dd'T'HH:mm:ss.SSSSSSS'Z'"]
          target => "@timestamp"
        }
        mutate {
          gsub => [
            "message", "--------------- Event Log Start Here ---------------\n","",
            "message", "\n--------------- Event Log End Here ---------------", ""
          ]
          remove_field => ["logtime", "Message"]
        }
  }
}
output {
#  if "_grokparsefailure" in [tags] {
#    # write events that didn't match to a file
#    file { "path" => "/opt/logstash/grok_failures.txt" }
#   } else {
     elasticsearch {
       hosts => "10.54.52.31:9200"
       user => "logstash_writer"
       password => "lastmile"
       manage_template => false
       index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
       #document_type => "%{[@metadata][type]}"
     }
#   }
}
6

Sheesh, OK this works. Seems like I didn't need the ".keyword"!

  if [app_id] == "LoggingService" {
7

I believe you have to use [app_id][keyword] instead of [app_id.keyword] in Logstash config to refers to a child field.

8

[app_id] is not a child field in my setup. Thanks.

9

It seems to me that you confused the .keyword multi-field in ES mappings with the actual field name being processed in Logstash?

10

Correct.