# Elasticsearch performance in HDD vs SSD and 32 GB vs 64 GB of RAM

**URL:** https://discuss.elastic.co/t/elasticsearch-performance-in-hdd-vs-ssd-and-32-gb-vs-64-gb-of-ram/334622
**Category:** Elasticsearch
**Created:** [May 29, 2023, 9:37pm UTC](https://discuss.elastic.co/t/elasticsearch-performance-in-hdd-vs-ssd-and-32-gb-vs-64-gb-of-ram/334622 "2023-05-29T21:37:39Z")
**Posts on this page:** 6
**Page:** 2

<div class="post-metadata">

### Author: ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)
#### Post date: [June 2, 2023, 8:45am UTC](https://discuss.elastic.co/t/elasticsearch-performance-in-hdd-vs-ssd-and-32-gb-vs-64-gb-of-ram/334622/21 "2023-06-02T08:45:01Z")

</div>

> [@Don\_Boscow](#):
>
> Is there a ballpark figure for how much additional space the index can occupy on hard disk (in addition to being stored in 64 GB of RAM) when 2 TB of data is indexed in ES?

This blog post, while old, is still about right. TLDR disk usage will often be within a factor of 2 of the input data size (either way) depending on the data and the mappings

> **[Part 2.0: The true story behind Elasticsearch storage requirements](https://www.elastic.co/blog/elasticsearch-storage-the-true-story-2.0)**

> [@Don\_Boscow](#):
>
> In the same vein, if we are indexing images through vectors in ES

Don't store large binary data like images in ES, it's a waste of your valuable in-cluster resources. Put them somewhere cheaper, with a link stored in ES which points to the binary data. You can _index_ them if you want to use vector search, just don't _store_ them there. See [store | Elasticsearch Guide [8.11] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-store.html) for a little more info on the difference.

---

<div class="post-metadata">

### Author: ![Don\_Boscow](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/don_boscow/32/121568_2.png) [@Don\_Boscow](https://discuss.elastic.co/u/Don_Boscow)
#### Post date: [June 2, 2023, 9:01am UTC](https://discuss.elastic.co/t/elasticsearch-performance-in-hdd-vs-ssd-and-32-gb-vs-64-gb-of-ram/334622/22 "2023-06-02T09:01:42Z")

</div>

Thanks David. Yes, that sounds like a good idea. We can store the images in S3 or some other storage and do just the indexing, classifying elsewhere. Just curious - the accompanying Medium article (for some reason I am unable to paste the link here, the post is flagged) seemed very promising. This was obviously a basic idea, to make a proper image search engine, we have to employ a combination of sophisticated NLP, vector feature extraction, self-supervised classification, random forests, etc. Will all this be possible by just running the classification, feature extraction, etc. on ES while the actual images are stored in a different server, like S3, for serving to the end user?

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/6/c/6c75462417380af5974c8dbd6247fd6737ba8794.jpeg)

---

<div class="post-metadata">

### Author: ![Don\_Boscow](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/don_boscow/32/121568_2.png) [@Don\_Boscow](https://discuss.elastic.co/u/Don_Boscow)
#### Post date: [June 2, 2023, 9:03am UTC](https://discuss.elastic.co/t/elasticsearch-performance-in-hdd-vs-ssd-and-32-gb-vs-64-gb-of-ram/334622/23 "2023-06-02T09:03:45Z")

</div>

Thanks. I won't. I will stick to 50 GB shard sizes as you and others have explained in this post. I was using this just as an example. My concern is that server with SSD and high RAM (~64 GB) is very expensive, so I need a way to have an idea of how much space I might end up needing to index 5 PB of data (plus its replicas, assuming only one replica for now), so that I can estimate the potential cost of such an endeavour.

---

<div class="post-metadata">

### Author: ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)
#### Post date: [June 2, 2023, 9:40am UTC](https://discuss.elastic.co/t/elasticsearch-performance-in-hdd-vs-ssd-and-32-gb-vs-64-gb-of-ram/334622/24 "2023-06-02T09:40:39Z")

</div>

> [@Don\_Boscow](#):
>
> Will all this be possible by just running the classification, feature extraction, etc. on ES while the actual images are stored in a different server, like S3, for serving to the end user?

Not sure, this is outside my area of expertise. I would hope so, but you might do better to open a separate thread on this question because the experts in this area have probably stopped reading this thread by now.

---

<div class="post-metadata">

### Author: ![Don\_Boscow](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/don_boscow/32/121568_2.png) [@Don\_Boscow](https://discuss.elastic.co/u/Don_Boscow)
#### Post date: [June 2, 2023, 9:47am UTC](https://discuss.elastic.co/t/elasticsearch-performance-in-hdd-vs-ssd-and-32-gb-vs-64-gb-of-ram/334622/25 "2023-06-02T09:47:22Z")

</div>

Got it. Thanks a lot, David. You opened up my mind towards an area I wouldn't have thought of by myself! 🙂

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [June 30, 2023, 9:48am UTC](https://discuss.elastic.co/t/elasticsearch-performance-in-hdd-vs-ssd-and-32-gb-vs-64-gb-of-ram/334622/26 "2023-06-30T09:48:00Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

[Previous page](https://discuss.elastic.co/t/elasticsearch-performance-in-hdd-vs-ssd-and-32-gb-vs-64-gb-of-ram/334622.md?page=1)
