Force logging-and-metrics cluster to run curator job

[JPT (Jan-Patrick Thurk)]

Hello,

we had the situation that our logging-and-metrics cluster get a lot of logs, because some trouble with a cluster. This than resulted in two of three nodes had 100% disk space usage. We tried to restart the cluster but afterwards only a bit more space was free. One node was still on 100% disk usage. So we decided to configure temporary a shorter retention rate:

./elastic-cloud-enterprise.sh set-logging-and-metrics-policy --pattern service-logs-* --days 5
./elastic-cloud-enterprise.sh set-logging-and-metrics-policy --pattern proxy-logs-* --days 5

The problem was now that we had to wait for the automatic job to clean the older logs in the night.

Setup:
ECE 2.2

Now my question:
Is there any possibility to manually trigger the retention clean-up job? It is very unsatisfying to wait for the job to run automatically.

Thank you!

Jan

[Yuri (Yuri Tceretian)]

Hello @JPT,
unfortunately that is not possible. However, you can delete indices by sending delete requests to the cluster (there is a handy tool in the admin console UI called Console API. It allows you to send requests to the cluster).
Also, you can temporarily add more disk space to a particular node or set of nodes, via "override disk quota"

[JPT (Jan-Patrick Thurk)]

Hello Yuri,

thank you for the explanation. The workaround with temporarily override the disk quota is useful.
But deleting the indices manual is not satisfying, because the risk to make a mistake in a manual task is given.
Will the behavior be different, when the loggin-and-metrics cluster will be on version 7?
Could we then use the Console API to trigger the index lifecycle management?

BR,
Jan

[Yuri (Yuri Tceretian)]

ECE 2.2 allows you to upgrade logging cluster. So, If you upgrade that cluster to a version (6.7.0+) that supports index lifetime management, you can use that feature but keep in mind that the curation embedded into ECE will still work too.