How much cpu power needed for elk consider security use case?

Elastic Security
1

elk documentation say that for every 20 shard we need 1 gb ram what about cpu?

2

Sizing of Elasticsearch depends on the use case and data volumes ingestred and how long this data is retained. The data point you mentioned above is out of data and can not be used for sizing.

If you provide some information about how much data you will be ingesting per day and how long you will need to keep this data in the cluster someone may be able to give some pointers. It is also important to know what types of hardwarre you plan to deploy on as storage performance often tend to be the main limiting factore rather than CPU and RAM.

If you are looking for CPU/RAM ratios and type of storage for different node types I would recommend looking at the ratios used on Elastic Cloud as they generally follow best practices.

1 Like
3

Thank you for your reply.
I want to ask if there something like equation that I can depend on.
Let's assume I ingest 1gb of data every day and i want to keep it for one month what calculations do you use.

4

The first thing to estimate is how much you will be indexing each day and the total amount of space this data will take up on disk. How much space data will take up on disk compared ton the raw size of the data will depend on the format of the vdata and the mappimgs and index settings used. A simplified assumption that can be used as a starting point is that it takes up the same amount of space as the raw data. If you need high availability you will also need a replica shard, which doubles the size on disk.

Once you know this, the scale will affect the cluster architecture you use. For small use cases a simple 3 node cluster where all nodes have all roles may be sufficient. If your daily data volumes are larger and/or you have a long retention period a hot-warm architecture may make sense. In such architecture you have different types of data nodes with diffrerent profiles. For very large data volumes it is even possible a hot-warm-cold cluster architecture may make sense.

1 Like
5

Thank you, I apperciate your reply

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.