# Bigdata storage of data
**URL:** https://discuss.elastic.co/t/bigdata-storage-of-data/136784
**Category:** Elasticsearch
**Created:** [June 21, 2018, 5:21am UTC](https://discuss.elastic.co/t/bigdata-storage-of-data/136784 "2018-06-21T05:21:46Z")
**Posts on this page:** 13
**Page:** 1
### Author:  [@gayathri](https://discuss.elastic.co/u/gayathri)
#### Post date: [June 21, 2018, 5:21am UTC](https://discuss.elastic.co/t/bigdata-storage-of-data/136784/1 "2018-06-21T05:21:46Z")
Hi Team,
Do we have any options to store the data in Bigdata platform in the case of large volume of data per day
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [June 21, 2018, 5:26am UTC](https://discuss.elastic.co/t/bigdata-storage-of-data/136784/2 "2018-06-21T05:26:12Z")
What big data platform are you referring to?
---
### Author:  [@gayathri](https://discuss.elastic.co/u/gayathri)
#### Post date: [June 21, 2018, 7:08am UTC](https://discuss.elastic.co/t/bigdata-storage-of-data/136784/3 "2018-06-21T07:08:06Z")
Hi Team,
We need to store the data in 2TB per day basis. So we don't think Elasticsearch will be able to suffice the requirement. Is there a way to collaborate the system to any bigdata storage ?
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [June 21, 2018, 7:13am UTC](https://discuss.elastic.co/t/bigdata-storage-of-data/136784/4 "2018-06-21T07:13:17Z")
I know of Elasticsearch clusters capable of handle considerably more than 2TB of ingest per day. How long are you going to store the data? How long do you need it searchable in Elasticsearch?
What is the use case?
---
### Author:  [@gayathri](https://discuss.elastic.co/u/gayathri)
#### Post date: [June 21, 2018, 7:23am UTC](https://discuss.elastic.co/t/bigdata-storage-of-data/136784/5 "2018-06-21T07:23:30Z")
Hi ,
We need to Keep 3 months of data mainly this comes as batch application logs.
We have ony 500GB internal storage. How can we store the large amount of data? Do we need external disk or any bigdata storage, can you suggest on this ?
Is there any estimate of the hardware needed ? (# of nodes,
vCPUs, RAM & storage per node)
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [June 21, 2018, 7:31am UTC](https://discuss.elastic.co/t/bigdata-storage-of-data/136784/6 "2018-06-21T07:31:20Z")
If you are going to query data through Elasticsearch, the data has to reside in Elasticsearch.
If we assume the size of the data stays the same once indexed (it can grow or shrink depending on a number of factors) and that you will have 1 replica, you will need at least 120TB of storage, so 500GB will not go far.
I would recommend looking at the following resources:
> **[How many shards should I have in my Elasticsearch cluster?
| Elastic](https://www.elastic.co/blog/how-many-shards-should-i-have-in-my-elasticsearch-cluster)**
>
> Elasticsearch is a very versatile platform, that supports a variety of use cases, and provides great flexibility around data organisation and replication strategies. This flexibility can however somet...
[https://www.elastic.co/elasticon/conf/2016/sf/quantitative-cluster-sizing](https://www.elastic.co/elasticon/conf/2016/sf/quantitative-cluster-sizing)
[https://www.elastic.co/webinars/using-rally-to-get-your-elasticsearch-cluster-size-right](https://www.elastic.co/webinars/using-rally-to-get-your-elasticsearch-cluster-size-right)
---
### Author:  [@gayathri](https://discuss.elastic.co/u/gayathri)
#### Post date: [June 21, 2018, 9:57am UTC](https://discuss.elastic.co/t/bigdata-storage-of-data/136784/7 "2018-06-21T09:57:18Z")
Hi
We will not get more that 500GB internal storage in our system. Can you suggest how we can store the 2TB data per day for our application.
Do we have any option to integrate Elasticsearch with external data store. If so please suggest the methods
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [June 21, 2018, 10:01am UTC](https://discuss.elastic.co/t/bigdata-storage-of-data/136784/8 "2018-06-21T10:01:44Z")
What kind of external storage do you have in mind? SAN?
---
### Author:  [@gayathri](https://discuss.elastic.co/u/gayathri)
#### Post date: [June 21, 2018, 10:28am UTC](https://discuss.elastic.co/t/bigdata-storage-of-data/136784/9 "2018-06-21T10:28:05Z")
Can you please suggest on how to store the large amount of data per day and keep it for atleast 3 months, as we don't have much internal storage for our servers.
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [June 21, 2018, 10:28am UTC](https://discuss.elastic.co/t/bigdata-storage-of-data/136784/10 "2018-06-21T10:28:39Z")
I can't unless you answer my questions. You will need access to a good amount of storage as I outlined earlier. Do you have access to SAN that you can mount on the nodes?
---
### Author:  [@gayathri](https://discuss.elastic.co/u/gayathri)
#### Post date: [June 21, 2018, 10:41am UTC](https://discuss.elastic.co/t/bigdata-storage-of-data/136784/11 "2018-06-21T10:41:34Z")
Yes we have access to SAN. But that is also not suffice our requirement.
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [June 21, 2018, 10:44am UTC](https://discuss.elastic.co/t/bigdata-storage-of-data/136784/12 "2018-06-21T10:44:48Z")
You will need to get access storage and hardware somehow to make this work. Exactly how you do that does however not seem to be something we can help you with here. Have you considered hosting in the cloud?
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 19, 2018, 10:44am UTC](https://discuss.elastic.co/t/bigdata-storage-of-data/136784/13 "2018-07-19T10:44:54Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.