What does Warm-node do after Curator running?

Elastic Stack Elasticsearch
1

Hello,

I posted a similar post yesterday but nobody replied yet, so I am posting this topic as a simple question.

  • What does Warm-node do after Curator running? Are they performing RE-Indexing?

  • Warm-nodes keep holding CPU usage after old indices came via Curator? CPU usage is getting higher based on the indices size increase from Hot-nodes, and never goes down.

Please advise!

cpu%20(1)
cpu%20(1)1485×648 89.8 KB

2

What is the heap usage on the warm nodes like? Do you see any evidence of long or frequent GC in the Elasticsearch logs that could explain CPU usage? Are you querying the data on the warm nodes?

3

Thank you for your replies.

I am not sure if i am able to answer for your question correctly but willing to try.

Heap usage was about 100% when i checked with _cat query.

Can you please advise how to check GC status in the log? Fyi, my log level is set as default so i assume current level is INFO. Is it enough to see the GC status in the log?

Sorry the datacenter where i worked is really strick and cannot access from outside. I can provide more detail info next monday.

4

If it was close to 100%, it sounds like you are suffering from heap pressure. There are a few things you should look at:

  • Have you optimised your mappings in order to minimise the amount of field data being used? Default dynamic mapping maps string fields as both text and keyword. If you are only going to run aggregations against them and not free-text search, you can reduce heap usage significantly by mapping them as keyword only.
  • Having lots of small indices and shards can also be inefficient and drive up heap usage. have a look at this blog post for guidance on shards and sharding practices.

If you have already applied these recommendations, you may simply be at the point where you need to scale up or out.

5

Yes we already applied both recommendations. I will take a look again if there are more thing to do for #2. Also we are considering scale out next month.

Thank you a lot!!

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.