Capacity planning tools?

Elastic Stack Elasticsearch
1

Hello Elastic experts,

Is there any capacity planning tools? For example, given # of records to index and use cases (with SLA), how much H/W is needed?

thanks in advance,
Lin

2

This is extremely hard to do since there are too many factors. Ignoring the variety in hardware at least the following factors contribute:

  • Number of indexes.
  • Number of nodes.
  • Number of shards.
  • Number of documents per index.
  • Frequency of write operations.
  • Frequency of deletions.
  • Size of each document.
  • How documents are mapped.
  • Frequency of queries.
  • Types of queries.
  • Whether indexes are queried uniformly or if there are "hot spots".
3

Thanks Magnus,

If so, what is your suggested way of doing capacity planning? Doing real testing?

BTW, what do you mean "documents are mapped"?

regards,
Lin

4

Yes, I advocate real-world testing and being prepared to scale your cluster.

By "How documents are mapped" I mean how Elasticsearch analyzes (tokenizes) and indexes a text. This affects how many tokens (terms) are added to the index and together with the type of queries made it plays an important part of the cluster's query performance.

5

Thanks Magnus!

regards,
Lin