AppSearch disk consumption keeps growing

Elastic Search
1

Hello, I see my dockerized Enterprise Elastic Search v7.7.0 self hosted appsearch is consuming each time more and more disk space, even my document count is almost the same.

I even deleted the engine (30k records) but didn't see any change in the disk space after an hour. Then I made a full import of my documents and the disk space didn't change noticeabilly nither. I guess that's ES way to handle the filesystem.

After that I've been using appsearch normally, but it has increased 4gb in less than a week. I still have 30k documents.
Could it be the analytics indexes? Or some sort of transaction logs? In Kibana I see a lot of EES indexes.

How can I stop EES eating my storage? Or delete the analytics in order to free disk space?
Please help. I can't keep increasing my storage every week

2

This is my Kibana report for bigger indexes. As you can see, all of them regard logging and monitoring wich I don't need.

ScreenHunter_302 Jun. 07 17.14
ScreenHunter_302 Jun. 07 17.141869×836 192 KB

3

Any ideas? :confused:

4

Hi
I see a setting 'xpack.monitoring.history.duration' that can set retention duration in this doc. And other indices could be deleted automatically, I believe.

5

Thank you. That setting doesn't work with basic licence, so I'm stuck with 7d history of those indices. At least, they have their retention.

The other indices (.ent-search-api-ecs-logs-togo_production and .ent-search-app-search-analytics-ecs-logs-togo_production) still accumulate endlessly

6

Another way , use indices management tool like Elasticsearch Curator to manage indices automatically.

7

Thanks. I should check on that!

However the bottomline about this issue is whether I can or can't delete those EES indexes, since they are automatically created by this module, not being automatically deleted, and have no way to manage them in the EES console.

if I delete them, am I going to break something? Do I need to have any precautions regarding their age?

I guess next time my HD is about to fill, I'll find it out...

8

I deleted all those logs and everything seems to be working fine.

It would be nice to find a way to avoid them to be created in the first place :slight_smile:

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.