# How ElasticSearch indexes the content of the files which are already compressed

**URL:** <https://discuss.elastic.co/t/how-elasticsearch-indexes-the-content-of-the-files-which-are-already-compressed/29203>\
**Category:** Elasticsearch\
**Created:** [September 13, 2015, 7:56pm UTC](https://discuss.elastic.co/t/how-elasticsearch-indexes-the-content-of-the-files-which-are-already-compressed/29203 "2015-09-13T19:56:04Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![rohansjsu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rohansjsu/32/4749_2.png) [@rohansjsu](https://discuss.elastic.co/u/rohansjsu)\
**Post date:** [September 13, 2015, 7:56pm UTC](https://discuss.elastic.co/t/how-elasticsearch-indexes-the-content-of-the-files-which-are-already-compressed/29203/1 "2015-09-13T19:56:04Z")

</div>

I have multiple files which are compressed in .gz format and I need to have search indexing on the content of the files which are compressed. How does Elasticsearch performs this ?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [September 13, 2015, 8:03pm UTC](https://discuss.elastic.co/t/how-elasticsearch-indexes-the-content-of-the-files-which-are-already-compressed/29203/2 "2015-09-13T20:03:41Z")

</div>

You'll have to feed Elasticsearch uncompressed data.

---

<div class="post-metadata">

**Author:** ![rohansjsu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rohansjsu/32/4749_2.png) [@rohansjsu](https://discuss.elastic.co/u/rohansjsu)\
**Post date:** [September 13, 2015, 8:17pm UTC](https://discuss.elastic.co/t/how-elasticsearch-indexes-the-content-of-the-files-which-are-already-compressed/29203/3 "2015-09-13T20:17:47Z")

</div>

thanks,

Since I have huge amount of data (around 100 TB) I was considering to compress the data. Is there any other way without uncompressing the data for indexing. (like using tools such as Solr and Apache tika) ?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [September 13, 2015, 8:21pm UTC](https://discuss.elastic.co/t/how-elasticsearch-indexes-the-content-of-the-files-which-are-already-compressed/29203/4 "2015-09-13T20:21:12Z")

</div>

I'm not sure I understand the problem. You can still keep the data compressed while at rest, but at the exact moment you feed it to Elasticsearch via an API of some sort it must not be compressed. The uncompressed source data never needs to touch the disk though.

If you have 100 TB data I'd be more worried about the amount of horsepower (CPU, RAM, and disk) that it's going to take for Elasticsearch to index the data.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 11:50pm UTC](https://discuss.elastic.co/t/how-elasticsearch-indexes-the-content-of-the-files-which-are-already-compressed/29203/5 "2017-07-05T23:50:36Z")

</div>