Invalid_index_name_exception with variable name

Elastic Stack Logstash
1

I'm getting these warnings with the 6.3.0 stack:
[2018-06-26T12:45:22,619][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"logstash-%{Environment}-2018.06.26", :_type=>"doc", :_routing=>nil}, #LogStash::Event:0x54432d92], :response=>{"index"=>{"_index"=>"logstash-%{Environment}-2018.06.26", "_type"=>"doc", "_id"=>nil, "status"=>400, "error"=>{"type"=>"invalid_index_name_exception", "reason"=>"Invalid index name [logstash-%{Environment}-2018.06.26], must be lowercase", "index_uuid"=>"na", "index"=>"logstash-%{Environment}-2018.06.26"}}}}

in the input fields i have:
add_field => { "Environment" => "production" }

and the output is:
index => "logstash-%{Environment}-%{+YYYY.MM.dd}"

The error goes away when i change the "variable name" to environment instead of Environment.
I understand that the index name cannot be uppercase, but should the variablename also be affected by this?

2

The event didn't actually contain an Environment field.

3

@magnusbaeck

That's very strange. Since i only use logstash to pull data from AWS Cloudwatch logs and populate it with the add_field => { "environment" => "production" }. Why would it matter if i change the E from capital to lowercase.
If i change the environment to Environment i get the error. And it stops when i switch it back to environment.

4

That's very strange. Since i only use logstash to pull data from AWS Cloudwatch logs and populate it with the add_field => { "environment" => "production" }. Why would it matter if i change the E from capital to lowercase.

Does your configuration contain

add_field => { "environment" => "production" }

or

add_field => { "Environment" => "production" }

?

Field names are case-sensitive so make sure you spell the field name identically everywhere.

5

@magnusbaeck

Ok i don't know if i were unclear. But i've been running this config on 5.6.x without issues.
The configuration was then:

add_field => { "Environment" => "production" }

This gave me errors on 6.3.0 (as mentioned in the first post)

I just simply renamed Environment to environment in all my cloudwatch_logs statements and in the output statement, and everything now works without warnings

6

Figured out what what caused all this.
I've been using the elapsed plugin, and it turns out that if there is a timeout it will create a new event that didn't have the %{Environment} variable set, thus trying to create a new index. when i changed the Environment variable to lowercase it created the index with the name logstash-%{environment}-2018.06.28

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.