# What is the largest size that one node can hold? **URL:** \ **Category:** Elasticsearch\ **Created:** [November 20, 2024, 4:42pm UTC](https://discuss.elastic.co/t/what-is-the-largest-size-that-one-node-can-hold/370851 "2024-11-20T16:42:46Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![dsagent](https://avatars.discourse-cdn.com/v4/letter/d/d2c977/32.png) [@dsagent](https://discuss.elastic.co/u/dsagent)\ **Post date:** [November 20, 2024, 4:42pm UTC](https://discuss.elastic.co/t/what-is-the-largest-size-that-one-node-can-hold/370851/1 "2024-11-20T16:42:46Z")
Hello I have a lot of data and I want to store it in elasticsearch What is the largest size that one node can hold? ---
**Author:** ![jessgarson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jessgarson/32/129841_2.png) [@jessgarson](https://discuss.elastic.co/u/jessgarson)\ **Post date:** [November 20, 2024, 6:49pm UTC](https://discuss.elastic.co/t/what-is-the-largest-size-that-one-node-can-hold/370851/2 "2024-11-20T18:49:29Z")
Thanks for reaching out, @dsagent. I have a few follow up questions here: - What version are you using? - How are you hosting Elasticsearch? I'd also be interested in hearing more from you about what you are looking for here. Here are some other responses from this forum that may be helpful here: - [Iterating through large datasets in various orders efficiently](https://discuss.elastic.co/t/iterating-through-large-datasets-in-various-orders-efficiently/362743) - [Elasticsearch ingest large of data](https://discuss.elastic.co/t/elasticsearch-ingest-large-of-data/322099) ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [November 20, 2024, 8:27pm UTC](https://discuss.elastic.co/t/what-is-the-largest-size-that-one-node-can-hold/370851/3 "2024-11-20T20:27:14Z")
Recent version have made a lot of improvements in heap usage, so nodes can hold quite large data volumes. The more data a node holds the longer it generally takes to query the data. The practical limit therefore often depend on performance requirements and load patterns. What is your use case and constraints? ---
**Author:** ![dsagent](https://avatars.discourse-cdn.com/v4/letter/d/d2c977/32.png) [@dsagent](https://discuss.elastic.co/u/dsagent)\ **Post date:** [November 21, 2024, 6:40am UTC](https://discuss.elastic.co/t/what-is-the-largest-size-that-one-node-can-hold/370851/4 "2024-11-21T06:40:01Z")
> [@jessgarson](#): > > What version are you using? 8.15.3 ---
**Author:** ![dsagent](https://avatars.discourse-cdn.com/v4/letter/d/d2c977/32.png) [@dsagent](https://discuss.elastic.co/u/dsagent)\ **Post date:** [November 21, 2024, 6:40am UTC](https://discuss.elastic.co/t/what-is-the-largest-size-that-one-node-can-hold/370851/5 "2024-11-21T06:40:17Z")
> [@jessgarson](#): > > How are you hosting Elasticsearch? local ---
**Author:** ![dsagent](https://avatars.discourse-cdn.com/v4/letter/d/d2c977/32.png) [@dsagent](https://discuss.elastic.co/u/dsagent)\ **Post date:** [November 21, 2024, 6:45am UTC](https://discuss.elastic.co/t/what-is-the-largest-size-that-one-node-can-hold/370851/6 "2024-11-21T06:45:39Z")
> [@jessgarson](#): > > I'd also be interested in hearing more from you about what you are looking for here. > > Here are some other responses from this forum that may be helpful here: > > - [Iterating through large datasets in various orders efficiently](https://discuss.elastic.co/t/iterating-through-large-datasets-in-various-orders-efficiently/362743) > - [Elasticsearch ingest large of data](https://discuss.elastic.co/t/elasticsearch-ingest-large-of-data/322099) thank you I want to know how much the maximum storage the node can hold ---
**Author:** ![dsagent](https://avatars.discourse-cdn.com/v4/letter/d/d2c977/32.png) [@dsagent](https://discuss.elastic.co/u/dsagent)\ **Post date:** [November 21, 2024, 6:50am UTC](https://discuss.elastic.co/t/what-is-the-largest-size-that-one-node-can-hold/370851/7 "2024-11-21T06:50:27Z")
> [@Christian\_Dahlqvist](#): > > Recent version have made a lot of improvements in heap usage, so nodes can hold quite large data volumes. The more data a node holds the longer it generally takes to query the data. The practical limit therefore often depend on performance requirements and load patterns. Can I do 30TB in one node? ---
**Author:** ![dsagent](https://avatars.discourse-cdn.com/v4/letter/d/d2c977/32.png) [@dsagent](https://discuss.elastic.co/u/dsagent)\ **Post date:** [November 21, 2024, 6:52am UTC](https://discuss.elastic.co/t/what-is-the-largest-size-that-one-node-can-hold/370851/8 "2024-11-21T06:52:50Z")
> [@Christian\_Dahlqvist](#): > > What is your use case and constraints? The use case is big, its data is many, there are no restrictions imposed, even if it needs additional resources. ---
**Author:** ![dsagent](https://avatars.discourse-cdn.com/v4/letter/d/d2c977/32.png) [@dsagent](https://discuss.elastic.co/u/dsagent)\ **Post date:** [November 21, 2024, 6:54am UTC](https://discuss.elastic.co/t/what-is-the-largest-size-that-one-node-can-hold/370851/9 "2024-11-21T06:54:14Z")
> [@dsagent](#): > > The use case is big, its data is many, there are no restrictions imposed, even if it needs additional resources. The important thing is that I store the data, so I asked when the node is tolerant ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [November 21, 2024, 7:15am UTC](https://discuss.elastic.co/t/what-is-the-largest-size-that-one-node-can-hold/370851/10 "2024-11-21T07:15:29Z")
When it comes to exactly how much data a node can hold it is usually limited by heap usage or performance requirements. How much heap is required to hold a specific amount of data on disk will primarily depend on your mappings. Certain features require more data to be stored on the heap and will therefore result in heap becoming a limiting factor faster. Indexing and querying will also require a certain amount of heap so what type of queries and how many you run concurrently will therefore also play a part. The only way to determine this correctly for your use case is to test with real data and queries. The more data you store on a node, the more disk I/O will likely be required to serve a query and latencies will generally go up as more data is added to the node, at least if you generally search large parts of the data and are not very targeted. It will also matter whether you run aggregations or searches bringing back documents as reading large number of documents from disk can result in a lot of random disk reads. Naturally the type of hardware, specifically storage, impacts this as well. When you run queries against the data, how targeted will these be (in terms of indices searched) and what kind of latencies are you willing to accept? Is this a simple search use case or will you be using more advanced mappings/features? ---
**Author:** ![dsagent](https://avatars.discourse-cdn.com/v4/letter/d/d2c977/32.png) [@dsagent](https://discuss.elastic.co/u/dsagent)\ **Post date:** [November 21, 2024, 7:25am UTC](https://discuss.elastic.co/t/what-is-the-largest-size-that-one-node-can-hold/370851/11 "2024-11-21T07:25:53Z")
I'm targeting a lot of them because I store a certain number of data in the cursor and then move to storage with another pointer As for the response time, I want it to be fast. Well, she'll tell you the amount of data. The data that I have reaches more than 15000 terabytes, so I want to store this data and search it ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [November 21, 2024, 7:30am UTC](https://discuss.elastic.co/t/what-is-the-largest-size-that-one-node-can-hold/370851/12 "2024-11-21T07:30:46Z")
> [@dsagent](#): > > I'm targeting a lot of them because I store a certain number of data in the cursor and then move to storage with another pointer I do not understand what you mean by this. Will each search target only a single index or a small set of indices or will you need to target most indices? > [@dsagent](#): > > As for the response time, I want it to be fast. If you want it to be fast you will need to use fast storage, e.g. local SSDs, and store a lot less data per node. For use cases with very high query concurrency or low latency requirements it is often necessary to ensure all data is cached in the operating system page cache in order to avoid disk I/O. It would also help if you could quantify what you mean by fast. ---
**Author:** ![dsagent](https://avatars.discourse-cdn.com/v4/letter/d/d2c977/32.png) [@dsagent](https://discuss.elastic.co/u/dsagent)\ **Post date:** [November 21, 2024, 7:34am UTC](https://discuss.elastic.co/t/what-is-the-largest-size-that-one-node-can-hold/370851/13 "2024-11-21T07:34:31Z")
> [@Christian\_Dahlqvist](#): > > Is this a simple search use case or will you be using more advanced mappings/features? Yes, I will use large mappings and change some values such as the length of the value 255 to 3000 And I don't know exactly what features I can use when I have started anything that can benefit me people with it ---
**Author:** ![dsagent](https://avatars.discourse-cdn.com/v4/letter/d/d2c977/32.png) [@dsagent](https://discuss.elastic.co/u/dsagent)\ **Post date:** [November 21, 2024, 7:49am UTC](https://discuss.elastic.co/t/what-is-the-largest-size-that-one-node-can-hold/370851/14 "2024-11-21T07:49:30Z")
> [@Christian\_Dahlqvist](#): > > I do not understand what you mean by this. Will each search target only a single index or a small set of indices or will you need to target most indices? I will target most indices ---
**Author:** ![dsagent](https://avatars.discourse-cdn.com/v4/letter/d/d2c977/32.png) [@dsagent](https://discuss.elastic.co/u/dsagent)\ **Post date:** [November 21, 2024, 7:56am UTC](https://discuss.elastic.co/t/what-is-the-largest-size-that-one-node-can-hold/370851/15 "2024-11-21T07:56:18Z")
> [@Christian\_Dahlqvist](#): > > and store a lot less data per node. How much TB ---
**Author:** ![dsagent](https://avatars.discourse-cdn.com/v4/letter/d/d2c977/32.png) [@dsagent](https://discuss.elastic.co/u/dsagent)\ **Post date:** [November 21, 2024, 7:57am UTC](https://discuss.elastic.co/t/what-is-the-largest-size-that-one-node-can-hold/370851/16 "2024-11-21T07:57:16Z")
> [@Christian\_Dahlqvist](#): > > It would also help if you could quantify what you mean by fast. I mean the speed of response in redoing the search results ---
**Author:** ![dsagent](https://avatars.discourse-cdn.com/v4/letter/d/d2c977/32.png) [@dsagent](https://discuss.elastic.co/u/dsagent)\ **Post date:** [November 21, 2024, 8:00am UTC](https://discuss.elastic.co/t/what-is-the-largest-size-that-one-node-can-hold/370851/17 "2024-11-21T08:00:33Z")
If I had 15000TB (15PT) how many nodes do I need? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [November 21, 2024, 8:20am UTC](https://discuss.elastic.co/t/what-is-the-largest-size-that-one-node-can-hold/370851/18 "2024-11-21T08:20:49Z")
I do not know. You need to test. I do not even know if your requirements are realistic. With that much data you will likely need multiple clusters as there are practical limits to cluster size. This depends on use case. If you are expecting very fast queries that target most indices and retrieve a large number of hits, potentially at a high query concurrency, I expect you are in trouble and will struggle to find a cost effective solution. ---
**Author:** ![dsagent](https://avatars.discourse-cdn.com/v4/letter/d/d2c977/32.png) [@dsagent](https://discuss.elastic.co/u/dsagent)\ **Post date:** [November 21, 2024, 8:35am UTC](https://discuss.elastic.co/t/what-is-the-largest-size-that-one-node-can-hold/370851/19 "2024-11-21T08:35:31Z")
> [@Christian\_Dahlqvist](#): > > I do not even know if your requirements are realistic. Yes the requirements are realistic and based on study ---
**Author:** ![dsagent](https://avatars.discourse-cdn.com/v4/letter/d/d2c977/32.png) [@dsagent](https://discuss.elastic.co/u/dsagent)\ **Post date:** [November 21, 2024, 8:38am UTC](https://discuss.elastic.co/t/what-is-the-largest-size-that-one-node-can-hold/370851/20 "2024-11-21T08:38:32Z")
> [@Christian\_Dahlqvist](#): > > With that much data you will likely need multiple clusters as there are practical limits to cluster size. This depends on use case. What are these limits? [Next page](https://discuss.elastic.co/t/what-is-the-largest-size-that-one-node-can-hold/370851.md?page=2)