Indices eating too much space (~50 GB each)

[The-Big-K]

I'm using Elasticsearch 7.5.2 on uBuntu. Recently, I began using elasticsearch to display relevant search results on every page load. This shot up the volume; but I also found out that it has created large index files. Note that I'm using 'app-search' to power my queries.

Here's the sample index files that are occupying too much space:

.app-search-analytics-logs-loco_togo_production-7.1.0-2020.01.26 => 52 GB
.app-search-analytics-logs-loco_togo_production-7.1.0-2020.01.27 => 53 GB

I tried deleting these using CURL, but they reappear and show lesser space (~5 GB each).

Following is the command I used -

curl -XDELETE -u myUserName:MyPassWord "http://localhost:9200/.app-search-app-logs-loco_togo_production-7.1.0-2020.01.25"

throws following error -

"root_cause":[{"type":"index_not_found_exception","reason":"no such index

I want to know if there is a way to control these indexes. I'm not sure what purpose do these indices solve and if there is a way to prevent them?

Thank you in advance.

[orhantoy]

Hi @The-Big-K

We are aware of the potential storage difficulties with analytics and API logs and we are planning on addressing those in the near future. Until then the only option is unfortunately to manually delete the older indices. You can use the date in the index name to determine which are older.

[The-Big-K]

Thank you for the response @orhantoy. Okay, is there any option to turn this off? Can ILM be effective here? If yes, how?