Slow Cluster in Elastic Cloud since updating to 7.12

Elastic Stack Elasticsearch
21

Hi @mom-douellet, thanks for the information.

Indeed the problem looks very similar to yours (100% cpu always, data inter node about 10 TB! And we ingested like only 8GB in the interval). I'm not in my working computer right now but I will provide a view of what happened:

After nobody from elastic mod team could help here, I had to "pay to play". As I am the only one managing the cluster I upgraded my license to platinum and filled an urgent ticket. My cluster was green but useless in this meantime, and I couldn't wait three business days, the SLA for normal things.

The engineers were very kind, provided two fixes during one day, and sent me a bunch of commands to run in dev tools a few times (4 hours interval, run the command).
And then the elastic dev team solved the issue and fixed the cluster.

I think you should do a rollback if you can (I couldn't find this option in elastic cloud). Probably elastic will provide a wider fix in the next days/weeks for the public. Hope the best for your cluster.

1 Like
22

Yeah, not sure I can rollback easily, but I'm in the fortunate position to be able to scrap the cluster and restart a new one.

Will search for the answer a couple of days. Maybe the dev team will push the fix publicly.

Take care

23

If you have created any Spaces you could be hitting this issue in Kibana; [Search Sessions] Kibana fails to update or delete sessions in non-default space · Issue #96124 · elastic/kibana · GitHub

You can check with a query like this;

GET .kibana_task_manager/_search
{ "size": 100 }

A fix is coming in 7.12.1. There are some work-around steps you can take described in that issue.

Regards,
Lee

24

Thanks @LeeDr this was spot on.

The work around did resolve the CPU and network issue.

25

Workaround did not resolve the performance issue. 7.11.2 Was fast. 7.12.0 is not.
When data is cached it is sometimes fast.
Even some Markdown text which does not require any further query result takes long to render.

1 Like
26

A huge thanks for this discussion. As a newbie I was very interesting to read

1 Like
27

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.