Deletion of indexed documents while doing the indexing with action set to index

Elastic Stack Logstash
1

I have seen a peculiar behavior while using the logstash JDBC input plugin and Elasticsearch output plugin.

Initial indexing has been completed and was trying to do the partial update for few columns.
When I have sent only few columns (Not all) for updating the existing document in elasticsearch, I see that those documents are getting deleted. This is with the default value of index for action parameter of elasticsearch output.

Ultimately I have used action as update to have the partial update done successfully.

But it is very strange to see that documents gets deleted when few fields have been sent to elasticsearch with action set to index.

Anyone has faced this issue? Any idea why this behavior. (Version 7.6.2)

2

If you are using "index" and setting the document_id then a new document will completely overwrite an existing document. Any columns you do not supply in the new document will not be saved from the old document.

3

Yes. That was what I was expecting when action is set to index. But when few of the columns are not supplied the documents are getting deleted from the index.

4

Can someone explain the behavior when action is set to index and action is set to update, how these requests will be treated internally?

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.