# Trying to push 17GB folder size of different log file into ES

**URL:** <https://discuss.elastic.co/t/trying-to-push-17gb-folder-size-of-different-log-file-into-es/55293>\
**Category:** Elasticsearch\
**Created:** [July 12, 2016, 1:14pm UTC](https://discuss.elastic.co/t/trying-to-push-17gb-folder-size-of-different-log-file-into-es/55293 "2016-07-12T13:14:42Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![devibala](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/devibala/32/70542_2.png) [@devibala](https://discuss.elastic.co/u/devibala)\
**Post date:** [July 12, 2016, 1:14pm UTC](https://discuss.elastic.co/t/trying-to-push-17gb-folder-size-of-different-log-file-into-es/55293/1 "2016-07-12T13:14:42Z")

</div>

Trying to push 17GB folder size of different log file into ES 8GB centos machine.

What are the best way to adapt for this .  
I did the following

Added /usr/share/elasticsearch/bin/elasticsearch ES\_HEAP\_SIZE = 2gb  
Added elasticsearch.yml = index.codec: best\_compression  
/etc/sysconfig/elasticsearch MAX\_OPEN\_FILES=65535 MAX\_LOCKED\_MEMORY=unlimited  
/etc/security/limits.conf elasticsearch - nofile 65535 elasticsearch - memlock unlimited

I am trying to push one entire log file into one document like that i need to push all the 17 GB data in ES under one index  
Observation :  
After some time ES status goes to red and data stop populating into ES from LS.  
Kindly help in understanding the memory allocation in ES ??

---

<div class="post-metadata">

**Author:** ![thiago](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thiago/32/32096_2.png) [@thiago](https://discuss.elastic.co/u/thiago)\
**Post date:** [July 12, 2016, 6:11pm UTC](https://discuss.elastic.co/t/trying-to-push-17gb-folder-size-of-different-log-file-into-es/55293/2 "2016-07-12T18:11:51Z")

</div>

Hello Devi,

I am trying to understand what are you trying to do. Are trying to index a single log file of 17gb???

Cheers

---

<div class="post-metadata">

**Author:** ![devibala](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/devibala/32/70542_2.png) [@devibala](https://discuss.elastic.co/u/devibala)\
**Post date:** [July 13, 2016, 12:55pm UTC](https://discuss.elastic.co/t/trying-to-push-17gb-folder-size-of-different-log-file-into-es/55293/3 "2016-07-13T12:55:12Z")

</div>

No its not single log file .Folder contains multiple sub directories and it stores different log file

---

<div class="post-metadata">

**Author:** ![thiago](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thiago/32/32096_2.png) [@thiago](https://discuss.elastic.co/u/thiago)\
**Post date:** [July 13, 2016, 1:16pm UTC](https://discuss.elastic.co/t/trying-to-push-17gb-folder-size-of-different-log-file-into-es/55293/4 "2016-07-13T13:16:20Z")

</div>

Ok Devi, could you please elaborate what do you mean by:

> [@devibala](#):
>
> I am trying to push one entire log file into one document

To understand what's going on with your elasticsearch node, send some logs from elasticsearch from when it goes red.

Regards

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 10:36pm UTC](https://discuss.elastic.co/t/trying-to-push-17gb-folder-size-of-different-log-file-into-es/55293/5 "2017-07-05T22:36:04Z")

</div>