# Best practice for storing big (~500 MB) files in ES

**URL:** <https://discuss.elastic.co/t/best-practice-for-storing-big-500-mb-files-in-es/72998>\
**Category:** Elasticsearch\
**Created:** [January 27, 2017, 10:39am UTC](https://discuss.elastic.co/t/best-practice-for-storing-big-500-mb-files-in-es/72998 "2017-01-27T10:39:32Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![janpom](https://avatars.discourse-cdn.com/v4/letter/j/e9a140/32.png) [@janpom](https://discuss.elastic.co/u/janpom)\
**Post date:** [January 27, 2017, 10:39am UTC](https://discuss.elastic.co/t/best-practice-for-storing-big-500-mb-files-in-es/72998/1 "2017-01-27T10:39:32Z")

</div>

I would like to store binary data chunks of up to ~500 MB in ES. These are not meant to be analyzed and used for searching. I merely want to store them and be able to retrieve them later. The reason for storing in ES is that the data is related to documents indexed in ES and I want to have everything in one place.

Now, I have figured that by using the binary data type (so that the data is not analyzed) and by increasing the http.max\_content\_length limit, I can upload and retrieve data chunks of sufficient size.

Other approach I have tried that didn't require messing up with http.max\_content\_length was to split the data into smaller chunks (10 MB each) and store them as multiple records in ES. This is a bit clunkier but it also worked.

I wonder if any of the two approaches is better than the other wrt performance or potential issues; or if there's any other best practice for storing big files in ES.

Thank you.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [January 27, 2017, 11:25am UTC](https://discuss.elastic.co/t/best-practice-for-storing-big-500-mb-files-in-es/72998/2 "2017-01-27T11:25:33Z")

</div>

Elasticsearch is totally not designed for that. I'd love that support of blobs could happen but I don't think it will ever happen.  
I'd probably collocate my blobs in another system like hdfs, couchdb, couchbase, MapR or whatever you prefer but storing 500mb of blobs in Elasticsearch is going to be a nightmare IMO.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 24, 2017, 11:25am UTC](https://discuss.elastic.co/t/best-practice-for-storing-big-500-mb-files-in-es/72998/3 "2017-02-24T11:25:51Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
