How much data for a warm node

Elastic Stack Elasticsearch
1

I'm researching the impossible questions for planning a cluster.

We need to keep some data for a year or more. If we do a hot/warm, the question is can we keep 10-20G or more on a warm node? Disk is cheap, nodes aren't.

This data would be daily log data, so would be "skipped" in almost all searches. I'm afraid the heap size is the limiting factor. These indices would be forecemerged and set to read-only before migrating to warm nodes.

I think if we close indices, we could use a lot more storage, but then we would have to handle requests to open them for searching. Murphy's law says they would want them all open at once anyway.

Any thoughts?

2

Heap space can often be the limiting factor when it comes to determining how much data you can store per node. Even though this webinar primarily discusses cold nodes, you should follow the same optimisation guidelines in order to get the most from warm nodes as well. Also make sure you follow best practices around sharding.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.