Kibana data suddenly got missing!

[Souvik_Das]

Today morning when I've tried to open my Kibana, I found the following error in my browser.

Then I've restarted Kibana. But after the restart, all of my indexes have been gone.

After that, to view all my works such that dashboards, visuals, I've imported my Saved Object backup file that is in json format.
After imported, I found all the visuals and dashboards are there as it is but I've lost past data from elasticsearch. I found data only from yesterday's 00:00 hour to the current date-time.
Why this type of sudden data loss has happened?
Can I somehow track the incident from a log or somewhere?
I've read a few posts in the elastic blog that refer to the same kind of issue.
Is that that a security threat?
Please help guys!!

[dadoonet (David Pilato)]

Could you share your full elasticsearch logs?

@Souvik_Das

Is that that a security threat?

Is your cluster accessible on internet? Or within your company? But without at least the default built-in security?

[Souvik_Das]

Yes it is accessible on the internet and I have tried to secure it using the https protocol.
If I want to see the elasticsearch logs, how to do that?
Please guide.

[dadoonet (David Pilato)]

@Souvik_Das

I have tried to secure it using the https protocol

HTTPS just encrypts communication over http. It does not prevent anyone to access your server.

@Souvik_Das

If I want to see the elasticsearch logs, how to do that?

How did you install elasticsearch? The installation guide tells what is the default dir depending on the package you installed.

[dadoonet (David Pilato)]

You should read https://www.elastic.co/blog/how-to-prevent-elasticsearch-server-breach-securing-elasticsearch

[Souvik_Das]

Thank you so much for your response. I'll go through the link to resolve the issue at my end.

[dadoonet (David Pilato)]

Note that the easiest way to have everything running and secured is just to start a cluster from cloud.elastic.co. You have 14 days for free if you want to try it.

[Souvik_Das]

Hi David, Can I enable this feature in my existing system? as I've already done all of my work in this system and re work all the stuffs is bit difficult and time consuming for me.

[dadoonet (David Pilato)]

If you have elasticsearch 6.8 or 7.6, yes.
Otherwise you need to buy a license.

[Souvik_Das]

My Kibana version is 6.8.7
How can I do that?
Please suggest.

[dadoonet (David Pilato)]

I think that reading the documentation would help. See https://www.elastic.co/guide/en/elasticsearch/reference/6.8/secure-cluster.html

[Souvik_Das]

Okay..thanks David..

[Souvik_Das]

Hi David,
Today when I was reviewing the indexes, I found an abnormal index name in the index management.

I have not created that index. How that index came here?
Is that a security breach?
Has someone hacked the elasticsearch?
Please have a look at it and advice what should be done from my end.

[Christian_Dahlqvist]

@Souvik_Das

I have not created that index. How that index came here?

It looks like someone has accessed your cluster so it is clearly not secure. You need to secure the cluster and/or remove it from the internet.

@Souvik_Das

Is that a security breach?

Yes, although technically you do not have any security in place to breach.

@Souvik_Das

Has someone hacked the elasticsearch?

Yes.

[Souvik_Das]

Thanks for the advice Christian..I'll work on this..

[pjanzen (Paul Janzen)]

You really should remove that cluster and rebuild from scratch. I would never trust that cluster again.. you can restore the data in a new cluster if you have backups of only the data indices...

[Souvik_Das]

Hi, I've closed the cluster to the internet. Now elasticsearch can be reached locally since the elk stack reside on the same server now. is that a feasible solution?