Loosing data in elasticsearch

Elastic Stack Elasticsearch
21

This makes sense, this would be from the kafka elasticsearch connector as I don't have any delete function enabled.

22

Here is the index setting:

{"grits":{"settings":{"index":{"creation_date":"1493658495456","number_of_shards":"5","number_of_replicas":"1","uuid":"EZ6LmCbxS3WSXlt20Q8HXw","version":{"created":"2000099"}}}}}

Can't find anything on ttl

23

That is good, because it was deprecated for a reason. I notice that you have a lot of nested fields. As each of the nested documents is stored as a separate document in Elasticsearch and all documents are updated when the main document is, it can mean that updating a single document will lead to multiple documents being updated and shown as deleted in the stats.

24

For reference, related discussion : https://stackoverflow.com/questions/45018344/using-elasticsearch-generated-ids-in-kafka-elasticsearch-connector/45182181#45182181

25

Thanks @Christian_Dahlqvist and @warkolm, It seems the problem has been resolved by changing to a different elasticsearch connector.

26

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.