Elasticsearch on AWS I3

Elastic Stack Elasticsearch
1

Have just switched from running our logging cluster based around r3.xlarge to the new i3.xlarge.

Have been able to shut down a third of my nodes due to improved performance and am paying less per node for the i3.xlarge than I was for the r3.xlarge. Have halved my costs!

I3's really hit a sweet spot

1 Like
2

What is your backup strategy? How are you automating snapshot / restore? Contemplating a similar move, from m4.2xlarge (EBS).

3

We only run a copy of days worth of logs on the 'front'. All data from recently completed logs are backed up onto s3 via the cloud_aws plugin. Yes the storage is ephemeral, but we would have to lose multiple nodes to lose data. Also as the indexes are aware of which AWS zone they are in - loss of a zone would not see us lose data. But things can still happen, so a last resort, we have the original pre-logstash source files we can go to. Currently we are using rsyslog, but plan to switch to Kafka., which would also hold post-logstash data.

Older data goes to a r3.xlarge instance with lots of ebs disk space - we lose the replica shards and optimise indices at this point, but all data will have been backed up to s3 and the data is rarely used. We can cope with a relatively long availability of that data. Two further smaller nodes are used as master only nodes as we do not allow the nodes at the front to be masters

1 Like
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.