If-in statement does not work if array has one object [SOLVED]

Elastic Stack Logstash
1

Hi, I prepare this question and also find solution after few hours, so I decide to upload question and answer, maybe it will help somebody:

QUESTION:

I have Filebeat in k8s that sends logs to Logstash. I have namespace apps-special. In my ES, I create rollover index with name rollover-special-000001 and alias special. This is my Logstash pipeline filter:

filter {
if [kubernetes][namespace] in ["apps-special"] {
    mutate {
      add_field => { "[@metadata][es-index]" => "special" }
    }
  } else if [kubernetes][pod][name] {
    mutate {
      add_field => { "[@metadata][es-index]" => "default-%{[kubernetes][pod][name]}-%{+xxxx-ww}" }
    }
  }
}

Expectation:
I would expect all pods from namespace apps-special will be in one index named rollover-special-000001 (which will rollover after time/conditions). Logs from all other namespaces shoud be created as <name-of-the-pod>-<YEAR>-<WEEK>.

Result:
I have many indices with <name-of-the-pod>-<YEAR>-<WEEK> pattern. I have no idea why the if statement did not match my namespace.

My investgation:
I have other namespaces where the match works perfectly. I also check, if kubernetes metadata in messages has proper namespace. So from output:
GET /<index-name>/_search

I grep the namespace:

 jq '.hits.hits[]."_source".kubernetes.namespace' | sort | uniq
 "apps-special"

What i find out, my if statements with other namespaces works fine, but the list for matching indices is grater than 1.

[kubernetes][namespace] in ["namespace-1", "namespace-2", "namespace-3"]

Conclusion:
I think, I have the right logic in Logstash filter, but I have no idea why is Logstash unable to process it in right way. From my investigation, it looks like the array must be >= 2 but I am not sure. Am I missing something?

Versions:
Filebeat: v7.9.3
Logstash: v7.8.1

ANSWER:
It looks like if-in statement does not work if array has only one object (array.length = 1). When I duplicate my apps-special it starts working properly, so after my change, my filter looks like this:

filter {
if [kubernetes][namespace] in ["apps-special", "apps-special"] {
    mutate {
      add_field => { "[@metadata][es-index]" => "special" }
    }
  } else if [kubernetes][pod][name] {
    mutate {
      add_field => { "[@metadata][es-index]" => "default-%{[kubernetes][pod][name]}-%{+xxxx-ww}" }
    }
  }
}

My sources was:

I was not able to find it in documentation. I hope this will help somebody to save a few minutes/hours.

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.