Badly formatted index, after interpolation still contains placeholder

Elastic Stack Logstash
1

Hello, after migration to the 8.12.1 I've started getting error "Badly formatted index, after interpolation still contains placeholder". When I'm trying to process report with Filebeat. More interesting is I get this message on one type of reports and for the other it works..

Output config

output {
  if [type] == "aws" {
    elasticsearch {
      index => "test-report-%{[PK]}"    
      ecs_compatibility => v8
			hosts => "${ELASTIC_HOST}"
      #ssl => true
      #ssl_certificate_verification => false
      #api_key => "${ELASTIC_ID}:${ELASTIC_KEY}"
    }
  }
}

Filter where I create PK field

    grok {
      match => {
        "[log][file][path]" => ["(?:%{BASE10NUM:PK}-)"]
      }
    }

any ideas? Could be done by wrong mapping in Kibana?

2

This means that the field that you are using in your index name does not exist in the event.

For example, you have this as your index name test-report-%{[PK]}, so it will get the value of the field PK and replace it, but if the field PK does not exist in your document, this value will not be replaced and you will get this error.

You need to check your grok if it is really working.

Alternatively you can add this filter to populate the PK field in case it does not exist and you can validate what was the issue.

filter {
    if ![PK] {
        mutate {
            add_field => { "PK" => "no-pk" }
        }
    }
}

Then if the field PK does not exist in the document, it will be created with the value no-pk and your index in this case will be named test-report-no-pk.

3

Thank you for your response! Thanks to your input I was able to find and fix my issue.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.