# Request Entity Too Large when index file json has size large 100mb

**URL:** <https://discuss.elastic.co/t/request-entity-too-large-when-index-file-json-has-size-large-100mb/202760>\
**Category:** Elasticsearch\
**Created:** [October 9, 2019, 5:21am UTC](https://discuss.elastic.co/t/request-entity-too-large-when-index-file-json-has-size-large-100mb/202760 "2019-10-09T05:21:30Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Vo\_Van\_Huy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vo_van_huy/32/55461_2.png) [@Vo\_Van\_Huy](https://discuss.elastic.co/u/Vo_Van_Huy)\
**Post date:** [October 9, 2019, 5:21am UTC](https://discuss.elastic.co/t/request-entity-too-large-when-index-file-json-has-size-large-100mb/202760/1 "2019-10-09T05:21:30Z")

</div>

How to index file json has size large 100mb with fscrawler and elastic.  
I use fscrawler 2.6 and elastic 6.8.0.

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [October 9, 2019, 6:58am UTC](https://discuss.elastic.co/t/request-entity-too-large-when-index-file-json-has-size-large-100mb/202760/2 "2019-10-09T06:58:25Z")

</div>

Hey,

see default http max content length is 100MB, see [https://www.elastic.co/guide/en/elasticsearch/reference/7.4/modules-http.html](https://www.elastic.co/guide/en/elasticsearch/reference/7.4/modules-http.html)

You should however not increase that limit, but rather reduce your batches if possible. Hope that makes sense.

--Alex

---

<div class="post-metadata">

**Author:** ![Vo\_Van\_Huy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vo_van_huy/32/55461_2.png) [@Vo\_Van\_Huy](https://discuss.elastic.co/u/Vo_Van_Huy)\
**Post date:** [October 9, 2019, 7:11am UTC](https://discuss.elastic.co/t/request-entity-too-large-when-index-file-json-has-size-large-100mb/202760/3 "2019-10-09T07:11:00Z")

</div>

yes, I know. but my data is books, it has data large 100mb, It cannot be split into small files.  
I did changes heap settings. (-Xms10g, -Xmx-10g) and "indexded\_chars" : "100%"

---

<div class="post-metadata">

**Author:** ![Vo\_Van\_Huy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vo_van_huy/32/55461_2.png) [@Vo\_Van\_Huy](https://discuss.elastic.co/u/Vo_Van_Huy)\
**Post date:** [October 9, 2019, 8:13am UTC](https://discuss.elastic.co/t/request-entity-too-large-when-index-file-json-has-size-large-100mb/202760/4 "2019-10-09T08:13:05Z")

</div>

I fixed it.  
Done

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [October 9, 2019, 8:19am UTC](https://discuss.elastic.co/t/request-entity-too-large-when-index-file-json-has-size-large-100mb/202760/5 "2019-10-09T08:19:10Z")

</div>

Note that there is a difference between what FSCrawler collects and what it generates. If you don't store the source (BASE64 binary document), then hopefully the extracted content is much less than the source itself.

Also, if you have very big documents, you can change [https://fscrawler.readthedocs.io/en/latest/admin/fs/elasticsearch.html#bulk-settings](https://fscrawler.readthedocs.io/en/latest/admin/fs/elasticsearch.html#bulk-settings) and make sure it's always under `100mb`.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 6, 2019, 8:19am UTC](https://discuss.elastic.co/t/request-entity-too-large-when-index-file-json-has-size-large-100mb/202760/6 "2019-11-06T08:19:16Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.