Data loss in Elasticstack

Elastic Stack Elasticsearch
1

Data used to get loaded to Elastic search without any issue but from last 4 weeks onward we can see reduction in Data when checked in Kibana Dashboard.
Later to find out where the Data loss is happening we checked in Database for Data volume and there is no much difference in Data Volume.

We checked the traffic from Logstash to Oracle DB and there is no issue with that as well.

We have not seen any changes in the Pipelines or Configs or Database but still there is a reduction in Data volume.

Could you please let us know, how to find out where the data loss is occurring in Elastic-stack.

Either it is from Database to Logstash
or From Logstash to Elastic-search

Awaiting your reply,

Thanks & Regards,
Manjunath

2

What do you mean by data loss ?
Was the index removed? Only some documents?

Could you share your full elasticsearch logs?
Also. Are you running it on cloud, private network, as a service on cloud.elastic.co?
Is it accessible on internet? Which version?

3

Thank you David for moving this Post to proper channel.

What do you mean by data loss ?

Documents count or Volume reduced.

Was the index removed?

No Index exist as it is.

Only some documents?

Yes, Documents count is less compared to previous days.

Could you share your full elasticsearch logs?

We are 6 elasticsearch containers and it is load balance. Could you please let me know whether you need logs on all the containers?

Also. Are you running it on cloud, private network, as a service on cloud.elastic.co?

No we are not running it on clould, it is private network.

Is it accessible on internet?

Yes,

Which version?

Please find the Version details given below.

  "version" : {
    "number" : "5.6.3",
    "build_hash" : "1a2f265",
    "build_date" : "2017-10-06T20:33:39.012Z",
    "build_snapshot" : false,
    "lucene_version" : "6.6.1"
  },

Thanks & Regards,
Manjunath

4

Could you use the citation icon "and the code icon </> when you want to cite a previous answer and want to share code/logs?

That will make your post easier to read. Thanks!

I'm editing your post.

5

Did you secure it so no one can access to the service? (I'd recommend using cloud.elastic.co to have a proper secured service).

5.6.3

This is toooo old.
At least upgrade to 6.8 or better 7.7.1.

6

Thank you for update David

Did you secure it so no one can access to the service? (I'd recommend using cloud.elastic.co to have a proper secured service).

Yes, As this Service was set up by the another team and we are supporting it.

This is toooo old. At least upgrade to 6.8 or better 7.7.1.

Yes, It is being Planned to upgrade to 6.8 but it may take some time. But we need the solution for the Dataloss. Could you please help us to get the possible ways to validate or Verify that there is no data loss

Thanks & Regards
Manjunath

7

You should check the logs. May be some index operation were rejected so you have less documents in elasticsearch?

8

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.