Can't modify the @timestamp field via logstash

Elastic Stack Logstash
1

Hi,
I recently had to migrate data from one index to another and wanted to maintain the original timestamps as in the original index. I have used the date plugin to achieve the same and it seems to work when I pipe the output to a local JSON file on my computer but fails when I push it to Elasticsearch and preview the documents in Kibana. I have attached my Logstash config file, can someone help me understand why the intended change isn't taking effect in the database?

logstash-simple.conf:

input {
	elasticsearch {
		cloud_auth => "my_auth"
			cloud_id => "my_cloud_id"
			index => "logistic_index"
			query => '{ "_source": ["message"], "query": { "query_string": { "query": "\\[ELK\\]", "fields": ["message"] } } }'
	}
}

filter {
	mutate {
		gsub => [
			"message", ",", "",
			"message", "}", "",
			"message", "{", ""
		]
	}
	date {
		match => ["@timestamp", "MMM dd, yyyy @ HH:mm:ss.SSS"]
		target => "@timestamp"
	}
	kv { }
}

output {
	elasticsearch {
		cloud_auth => "my_auth"
			cloud_id => "my_cloud_id"
			index => "logistic_index_new"
	}
}
2

If you change the target to another field name like tmp_timestamp do you see that field with the correct value in Elasticsearch?

3

Yes, doing so was possible but since we were migrating indices, such a solution wouldn't suffice.

However, this issue was fixed by making a slight change in the query by deleting the source filter. Thanks for the help!