If conditional " in " doesn't work, but " == " does? I need the "in" conditional to work!

Elastic Stack Logstash
1

In my logstash .conf, filter section when I use "==" it works, but when I replace with "in" it doesn't. Further down in the script, I do use "in" with multiple values, and none of them work either. Am I missing something?

"==" succeeds

filter {
  if [app_id.keyword] == ["LoggingService"] {
        grok {

"in" fails

filter {
  if [app_id.keyword] in ["LoggingService"] {
        grok {

RHEL 7, Logstash 6.2.0 then updated to 6.2.3, still doesn't work.

2

IIRC, The right-hand-side of an in clause gets parsed as a field reference, not as a literal array as you're attempting.

Unfortunately, the grammar is ambiguous, and field references have a higher precedence than literal arrays, so there's not much we can do to fix it without breaking the situation when people do intentionally have a field reference there.

While it is certainly more verbose, using equality along-side or clauses is going to be your best bet:

if [field] == "a" or [field] == "b" {
  # ...
}
3

Thank you for explaining.

4

I made the change, but it only worked with the [] around my string, ["LoggingService"].

[app_id.keyword] == ["LoggingService"]
5

I just retried and this works:

  if [app_id.keyword] == ["LoggingService"] {

This doesn't

  if [app_id.keyword] == "LoggingService" {

Here's the whole file:

input {

  beats {
    port => 5044
#    congestion_threshold => "999999999"
  }
}

filter {
  if [app_id.keyword] == ["LoggingService"] {
        grok {
          patterns_dir => ["/etc/logstash/conf.d/patterns"]
          match => ["message", "%{GREEDYDATA:Message}Timestamp : %{TIMESTAMP_ISO8601:logtime}"]
        }
        date {
          match => ["logtime", "ISO8601", "yyyy-MM-dd HH:mm:ss.SSSS", "yyyy-MM-dd HH:mm:ss,SSS", "yyyy-MM-dd'T'HH:mm:ss.SSSSSSS'Z'"]
          target => "@timestamp"
        }
        mutate {
          gsub => [
            "message", "--------------- Event Log Start Here ---------------\n","",
            "message", "\n--------------- Event Log End Here ---------------", ""
          ]
          remove_field => ["logtime", "Message"]
        }
  }
}
output {
#  if "_grokparsefailure" in [tags] {
#    # write events that didn't match to a file
#    file { "path" => "/opt/logstash/grok_failures.txt" }
#   } else {
     elasticsearch {
       hosts => "10.54.52.31:9200"
       user => "logstash_writer"
       password => "lastmile"
       manage_template => false
       index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
       #document_type => "%{[@metadata][type]}"
     }
#   }
}
6

Sheesh, OK this works. Seems like I didn't need the ".keyword"!

  if [app_id] == "LoggingService" {
7

I believe you have to use [app_id][keyword] instead of [app_id.keyword] in Logstash config to refers to a child field.

8

[app_id] is not a child field in my setup. Thanks.

9

It seems to me that you confused the .keyword multi-field in ES mappings with the actual field name being processed in Logstash?

10

Correct.

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.