# Which Storage type to be used for ELK on AWS?
**URL:** https://discuss.elastic.co/t/which-storage-type-to-be-used-for-elk-on-aws/170047
**Category:** Elasticsearch
**Created:** [February 26, 2019, 4:48pm UTC](https://discuss.elastic.co/t/which-storage-type-to-be-used-for-elk-on-aws/170047 "2019-02-26T16:48:59Z")
**Posts on this page:** 12
**Page:** 1
### Author:  [@rahulkumar1](https://discuss.elastic.co/u/rahulkumar1)
#### Post date: [February 26, 2019, 4:49pm UTC](https://discuss.elastic.co/t/which-storage-type-to-be-used-for-elk-on-aws/170047/1 "2019-02-26T16:49:00Z")
[https://www.elastic.co/guide/en/elasticsearch/plugins/master/cloud-aws-best-practices.html](https://www.elastic.co/guide/en/elasticsearch/plugins/master/cloud-aws-best-practices.html)
From the above it is recommended to use Instance storage, but then data is lost in the below scenarios:
- The underlying disk drive fails
- The instance stops
- The instance terminates
What is the general practice, how can this be handled ?
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [February 26, 2019, 5:01pm UTC](https://discuss.elastic.co/t/which-storage-type-to-be-used-for-elk-on-aws/170047/2 "2019-02-26T17:01:24Z")
Do you plan to run only one node in your cluster?
It's recommended to have 3 nodes (at least 2 data/master nodes + one master only node). In which case you'll have a copy of the data in the other nodes.
---
### Author:  [@rahulkumar1](https://discuss.elastic.co/u/rahulkumar1)
#### Post date: [February 28, 2019, 10:54am UTC](https://discuss.elastic.co/t/which-storage-type-to-be-used-for-elk-on-aws/170047/3 "2019-02-28T10:54:11Z")
I have posted it as a questing in thread [https://discuss.elastic.co/t/regular-elk-stack-on-aws/169997](https://discuss.elastic.co/t/regular-elk-stack-on-aws/169997)
---
### Author:  [@rahulkumar1](https://discuss.elastic.co/u/rahulkumar1)
#### Post date: [March 6, 2019, 5:10pm UTC](https://discuss.elastic.co/t/which-storage-type-to-be-used-for-elk-on-aws/170047/4 "2019-03-06T17:10:52Z")
Storage Type to be used is EBS
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [March 6, 2019, 5:56pm UTC](https://discuss.elastic.co/t/which-storage-type-to-be-used-for-elk-on-aws/170047/5 "2019-03-06T17:56:06Z")
Up to you but this is not really what is recommended. Read [https://www.elastic.co/guide/en/elasticsearch/plugins/6.6/cloud-aws-best-practices.html](https://www.elastic.co/guide/en/elasticsearch/plugins/6.6/cloud-aws-best-practices.html)
---
### Author:  [@rahulkumar1](https://discuss.elastic.co/u/rahulkumar1)
#### Post date: [March 6, 2019, 8:27pm UTC](https://discuss.elastic.co/t/which-storage-type-to-be-used-for-elk-on-aws/170047/6 "2019-03-06T20:27:09Z")
Yes, you are right, but then with the EFS, there are performance issues.
---
### Author:  [@rahulkumar1](https://discuss.elastic.co/u/rahulkumar1)
#### Post date: [March 6, 2019, 8:29pm UTC](https://discuss.elastic.co/t/which-storage-type-to-be-used-for-elk-on-aws/170047/7 "2019-03-06T20:29:01Z")
and the next question what if I am using the EC2 Instance Storage ? Again if the instance dies, the data is gone
---
### Author:  [@gbrown](https://discuss.elastic.co/u/gbrown)
#### Post date: [March 7, 2019, 12:01am UTC](https://discuss.elastic.co/t/which-storage-type-to-be-used-for-elk-on-aws/170047/8 "2019-03-07T00:01:25Z")
With Instance Storage, you're right, you'll need to make use of the features provided by Elasticsearch to make sure your data isn't lost.
First, have multiple nodes, preferably at least 3, and make sure to set `index.number_of_replicas` on each index to 1 or more. This ensures that [Elasticsearch will keep copies of your data on multiple nodes](https://www.elastic.co/guide/en/elasticsearch/guide/2.x/replica-shards.html) - it will keep copies on `index.number_of_replicas + 1` nodes. The higher you set that setting on each index, the more nodes that can die without losing data.
Second, use [Snapshots](https://www.elastic.co/guide/en/elasticsearch/reference/6.6/modules-snapshots.html) to periodically take backups of your data to a storage service like S3. This will ensure that you can recover your data even if your entire cluster dies.
EBS does work, mostly, but be aware that Elasticsearch is sensitive to IO performance, and that EBS has IOPS limits not only per volume, but per EC2 instance - that is, for a given EC2 instance size, there's a [limit to how many IOPS that instance can perform, no matter how many IOPS are provisioned for the attached volume](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSOptimized.html), and going over your IOPS limits can cause very significant slowdowns in performance that are difficult to diagnose.
---
### Author:  [@rahulkumar1](https://discuss.elastic.co/u/rahulkumar1)
#### Post date: [March 9, 2019, 11:46am UTC](https://discuss.elastic.co/t/which-storage-type-to-be-used-for-elk-on-aws/170047/9 "2019-03-09T11:46:28Z")
Thanks Gorden !! Which EC2 Instance would be recommended ? M42xLarge ?
---
### Author:  [@gbrown](https://discuss.elastic.co/u/gbrown)
#### Post date: [March 11, 2019, 4:46pm UTC](https://discuss.elastic.co/t/which-storage-type-to-be-used-for-elk-on-aws/170047/10 "2019-03-11T16:46:18Z")
Which EC2 instance you should use depends almost entirely on the workload you'll have and your budget. I can't really give concrete guidance here other than in the most general terms:
A server that's running Elasticsearch and nothing else should typically top out at 64GB of RAM, a [little less than 32GB given to Elasticsearch's heap](https://www.elastic.co/blog/a-heap-of-trouble) and the rest left for the OS to use for filesystem caching, and if you need more performance beyond that, add more nodes rather than making existing nodes bigger.
Beyond that and the disk IO considerations I mentioned above, the most reliable way to determine your hardware needs is to benchmark. We have a custom tool for benchmarking Elasticsearch called [Rally](https://github.com/elastic/rally) that can be customized pretty easily to replicate your workload.
---
### Author:  [@rahulkumar1](https://discuss.elastic.co/u/rahulkumar1)
#### Post date: [March 11, 2019, 5:01pm UTC](https://discuss.elastic.co/t/which-storage-type-to-be-used-for-elk-on-aws/170047/11 "2019-03-11T17:01:22Z")
Thanks Gordon, I am actually using three Master nodes across two availability zones. from a workload point of view, It's a brand new setup. I would take an example of a retail store like "[amazon.com](http://amazon.com)". I would be expecting a similar kind of load in about two years. So my EC2s should be able to survive for at least three years. Does these EC2 instances hold true for Data/Coordinating Nodes as well ?
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [April 8, 2019, 5:01pm UTC](https://discuss.elastic.co/t/which-storage-type-to-be-used-for-elk-on-aws/170047/12 "2019-04-08T17:01:25Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.