# Best practices for Elastic DB deployment for 100TB data **URL:** \ **Category:** Elasticsearch\ **Created:** [March 6, 2022, 3:19pm UTC](https://discuss.elastic.co/t/best-practices-for-elastic-db-deployment-for-100tb-data/298918 "2022-03-06T15:19:52Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![noamsh88](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/noamsh88/32/100897_2.png) [@noamsh88](https://discuss.elastic.co/u/noamsh88)\ **Post date:** [March 6, 2022, 3:19pm UTC](https://discuss.elastic.co/t/best-practices-for-elastic-db-deployment-for-100tb-data/298918/1 "2022-03-06T15:19:52Z")
Hello, We are designing new product that based on Elasticsearch that suppose to have around 100TB data Deployment will be on-prem (bare metal with Linux OS installed on it) and would like to ask: What is best practices from resources deployment POV? I mean: How many hosts do we need for such amount of data using elastic? what is best deployment of elastic? containers? Thanks Noam ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [March 6, 2022, 4:42pm UTC](https://discuss.elastic.co/t/best-practices-for-elastic-db-deployment-for-100tb-data/298918/2 "2022-03-06T16:42:52Z")
What is the use case? What is the expected indexing and query throughput? Do you have any latency requirements? What type of data are you indexing? What type of queries will you be running? ---
**Author:** ![DineshNaik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dineshnaik/32/89308_2.png) [@DineshNaik](https://discuss.elastic.co/u/DineshNaik)\ **Post date:** [March 6, 2022, 10:26pm UTC](https://discuss.elastic.co/t/best-practices-for-elastic-db-deployment-for-100tb-data/298918/3 "2022-03-06T22:26:45Z")
Also suggest you to take a look at [GitHub - gbaptista/elastic-calculator: Elasticsearch cluster calculator: How many shards and replicas should I have?](https://github.com/gbaptista/elastic-calculator) to get an idea . As @Christian_Dahlqvist mentioned knowing the use case and throughput would help in proper capacity calculation. ---
**Author:** ![noamsh88](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/noamsh88/32/100897_2.png) [@noamsh88](https://discuss.elastic.co/u/noamsh88)\ **Post date:** [March 7, 2022, 3:57pm UTC](https://discuss.elastic.co/t/best-practices-for-elastic-db-deployment-for-100tb-data/298918/4 "2022-03-07T15:57:46Z")
data will be sent to Elastic DB using Kafka, the data type will be json or logs from several systems regarding latency, not real time system, assuming it will up to 100ms regarding type of queries, Query DSL hope info is good enough for answer in high level, but do let me know if should provide more info ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [March 7, 2022, 8:35pm UTC](https://discuss.elastic.co/t/best-practices-for-elastic-db-deployment-for-100tb-data/298918/5 "2022-03-07T20:35:19Z")
It sounds like you will have time-based data in the form of logs coming in. Apart from that it does not tell me much. The following parameters will affect the sizing of your cluster, but it is not a complete list: - Number of different types of data being ingested. - Peak and average ingest rate for different data types. - The expected retention period of each data type. - Size and complexity of documents and associated mappings. - How you are querying the data. Are you using Kibana dashboards? Are you querying the data in other ways? Are queries targeting parts of the total data set or all data? - How many users/query concurrency do you expect to need to support? What are the latency requirements for dashboards and/or queries? - Hardware used, especially storage performance. I would recommend reading [this blog post](https://www.elastic.co/blog/benchmarking-and-sizing-your-elasticsearch-cluster-for-logs-and-metrics) and look at [this webinar](https://www.elastic.co/webinars/elasticsearch-sizing-and-capacity-planning). There are other resources available as well. In the end it is likely you will need to run some benchmarks in order to get an accurate estimate. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 4, 2022, 8:35pm UTC](https://discuss.elastic.co/t/best-practices-for-elastic-db-deployment-for-100tb-data/298918/6 "2022-04-04T20:35:55Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.