Index a document from Logstash using the ‘document_id

Elastic Stack Logstash
1

Hi
Basically we want to be able to index a document from Logstash using the ‘document_id’ parameter in the Elasticsearch output to overwrite\update a document and also use curator to delete anything older than 30days.

We have 2 options for this:

  1. Have a single index so the document_id parameter will function in our requirements were it overwrites\updates the existing document but the downfall of this is being able to delete indices older than 30days.
  2. Have a day base index to allow us to delete data older than 30 days but the document_id will only function if it is called on the same day. If a document is indexed the day after we get 2 documents with the same _id, one in todays and one in yesterdays.

Is there a known work around for this

Thanks

2

I think you're trying to have your cake and eat it. Document ids are only unique within a particular index, so if you want your document to be known by a single id you need a single index.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.