We have an Elastic stack cluster now upgraded to 7.4.2 ingesting about 600 GB of data per day running in Hot Warm architecture in our AWS account. We keep data for 90 days. Our Hot nodes are AWS i3en.2x large (8 VCPU, 64 GB RAM, 2x2.5 GB NVMe SSD, 25 Gbps network transfer) optimized for fast read/writes and Warm are d2.2xlarge (8 VCPU, 61 GB RAM, 6x2 GB HDD) optimized for disk storage and reasonable read/write.
a) We are planning to add some ingest only nodes to reduce the impact of spikes in ingestion on our Hot nodes. We plan to add these ingest only nodes in Auto scaling group to grow up and down as the load spikes. What machine types do you recommend for our ingest only nodes?
b) We plan to move to the Cold Node architecture and store older indices 30- 90 days in the cold nodes. What AWS machine specs makes sense for cold nodes?
Thank you for your help.
Regards,
Venkat