Elasticsearch Cluster Ingest Node Sizing

Elastic Stack Elasticsearch
1

Hi Team,

We're planning to add a new Ingest node to running ES cluster, this node is dedicated only for data ingestion. How to do capacity planning for ingest node, does this memory optimized or CPU optimized, to ingest Index size of around 1TB into the ES cluster (new Index weekly basis)? Please give me an advice.

Thank you in advance.

2

@chateesh I will say it depends. there are multipul fector you need to consider like

  • how many ingest_pipeline you have configured ?

  • data complexity

  • what enrich processor you are using ? if script then might be it will take more mory.

  • any many more ...

Please check this link for CPU optimized or memory optimized.

1 Like
3

Thankyou @sagarpatel

It has only one ingest pipeline.

4

@chateesh You have one pipeline but not sure how many processor it has ? Also, it will depnds on your index ration like per min how many documents are you indexing.

So my suggestion will be do POC with 1 week data and test load on node and based on that decide on node configuration.

Please check Elastic blog which will give you more understanding of benchmarking and sizing cluster.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.