# Could not index event to elasticsearch error

**URL:** <https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-error/267379>\
**Category:** Logstash\
**Created:** [March 16, 2021, 1:37pm UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-error/267379 "2021-03-16T13:37:04Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![atharvak](https://avatars.discourse-cdn.com/v4/letter/a/7ab992/32.png) [@atharvak](https://discuss.elastic.co/u/atharvak)\
**Post date:** [March 16, 2021, 1:37pm UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-error/267379/1 "2021-03-16T13:37:04Z")

</div>

Hello guys, I am getting following error about my filter and I have no clue why it is happening. I have included details.

### Filter

```auto
filter {
	if "cost_management" in [tags] {
		fingerprint {
			source => "message"
			target => "[@metadata][fingerprint]"
			method => "MURMUR3"
		}
		# Parsing of json events.
		json {
			source => "message"
			tag_on_failure => ["_grok_error_log_nomatch"]
		}
		# If line doesnt matched then drop that line.
		if "_grok_error_log_nomatch" in [tags] {
			drop { }
		}
		# Set timestamp as per input log events.
		date {
			# date format : 2019-12-23 23:28:42
			match => ["timestamp", "yyyy-MM-dd HH:mm:ss", "yyyy-MM-dd HH:mm:ss ", " yyyy-MM-dd HH:mm:ss", "ISO8601"]
			target => "@timestamp"
		}
		# Set index name as per given tags.
		mutate {
			add_field => ["index_name", "cost_management"]
		}
		mutate {
			convert => {
				"monthly_total_bill" => "float"
				"last_day_bill" => "float"
				"monthly_service_bill" => "float"
				"last_day_service_bill" => "float"
			}
		}
		# Remove unwanted tags from sorted logs.
		mutate {
			remove_tag => ["beats_input_codec_plain_applied", "beats_input_codec_plain_applied", "_grokparsefailure", "_geoip_lookup_failure", "multiline", "_jsonparsefailure"]
		}
	}
}

```

### Error

```auto
[2021-03-16T18:34:12,627][ERROR][logstash.outputs.elasticsearch][main] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>"3618653390", :_index=>"cost_management,cost_management-2021.03.16", :routing=>nil, :_type=>"_doc"}, #<LogStash::Event:0xa8069bb>], :response=>{"index"=>{"_index"=>"cost_management,cost_management-2021.03.16", "_type"=>"_doc", "_id"=>"3618653390", "status"=>400, "error"=>{"type"=>"invalid_index_name_exception", "reason"=>"Invalid index name [cost_management,cost_management-2021.03.16], must not contain the following characters [, \", *, \\, <, |, ,, >, /, ?]", "index_uuid"=>"_na_", "index"=>"cost_management,cost_management-2021.03.16"}}}}

```

### My Output Filter

```auto
output {
	file {
		create_if_deleted => true
		path => "/var/log/logstash/logstash_log"
		codec => "rubydebug"
	}
	elasticsearch {
		hosts => ["localhost:9200"]
		user => "admin"
		password => "passwd"
		manage_template => false
		index => "%{[index_name]}-%{+YYYY.MM.dd}"
		document_id => "%{[@metadata][fingerprint]}"
		http_compression => true
	}
}

```

* * *

What I have understood is that because of my output filter, logstash is inserting two index names separated by comma and elasticsearch is rejecting the indexing request. But I am unable to debug it. But my final index name should be `cost_management-2021.03.16` only.  
Please help me out. Thank you.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [March 16, 2021, 3:36pm UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-error/267379/2 "2021-03-16T15:36:12Z")

</div>

```
mutate { add_field => ["index_name", "cost_management"] }

"reason"=>"Invalid index name [cost_management,cost_management-2021.03.16], must not contain the following characters [, \", *, \\, <, |, ,, >, /, ?]", "index_uuid"=>"_na_", "index"=>"cost_management,cost_management-2021.03.16"}}}}

```

If that mutate gets executed twice, then [index\_name] will be an array, which the [code](https://github.com/elastic/logstash/blob/ea92b9d5294af12b2ee72f41c0477ea161474f4e/logstash-core/src/main/java/org/logstash/StringInterpolation.java#L89) will present as "cost\_management,cost\_management".

You could avoid this by using

```
mutate { replace => { "index_name" => "cost_management" } }

```

but you should really try to find why it is getting executed twice, since that may mean other things are being done twice. For example, if you point path.config at a directory then it will concatenate all the files in the directory into a single configuration. If the directory contains my.conf and my.conf.bak then the contents of both are part of the configuration.

---

<div class="post-metadata">

**Author:** ![atharvak](https://avatars.discourse-cdn.com/v4/letter/a/7ab992/32.png) [@atharvak](https://discuss.elastic.co/u/atharvak)\
**Post date:** [March 17, 2021, 5:33am UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-error/267379/3 "2021-03-17T05:33:48Z")

</div>

My bad. My `conf.d/` had same filter with different name. Removing that worked for me. Thank you very much.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 14, 2021, 5:34am UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-error/267379/4 "2021-04-14T05:34:33Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.