Number of Shards and replicas for HOT and WARM architecture


(Arvind Sharma) #1

Hi Team ,

I have been working on ELK stack from last few months , I m thinking to move my current architecture to hot and warm architecture .

Currently I have 5 servers with 48 gigs of ram's , 1 TB HDD and 8 Core processor with client and master on seprate nodes . ( with 5 shards and 2 replicas )

After lot of inspections I found that my disk is bottle next for indexing due to which my indexing is slow .

Disk writes almost busy all the times to 100% .

So I am thinking to take 2-3 SSD servers of 250 GB SSD , 48 gb ram and 16-32 core processor for HOT nodes and 3 nodes as warm nodes which will have 1TB HDD 32 GB RAM and 8 Core Processor ( For Cost optimization )

In my infra current logs are most searched , so HOT nodes should have replica's to make search in kibana fast .

So please suggest optimal number of nodes for HOT WARM architecture and thier configuration along with number of shards and replicas .

Thanks & Regards
Arvind


(system) #2