# OOME when using mapping **URL:** \ **Category:** Elasticsearch\ **Created:** [July 5, 2011, 4:12am UTC](https://discuss.elastic.co/t/oome-when-using-mapping/4775 "2011-07-05T04:12:29Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Anurag\_Phadke](https://avatars.discourse-cdn.com/v4/letter/a/5f9b8f/32.png) [@Anurag\_Phadke](https://discuss.elastic.co/u/Anurag_Phadke)\ **Post date:** [July 5, 2011, 4:12am UTC](https://discuss.elastic.co/t/oome-when-using-mapping/4775/1 "2011-07-05T04:12:29Z")
We are running a 5-node ElasticSearch 0.16.2 cluster, avg. size 20kb / document. However, I get an OutOfMemoryException on trying to insert 1million JSONS with the following mapping. curl -XPUT '[http://localhost:9200/socorro/](http://localhost:9200/socorro/)' -d ' index : number\_of\_shards : 30 number\_of\_replicas : 2 ' curl -XPUT '[http://localhost:9200/socorro/crash\_reports/\_mapping](http://localhost:9200/socorro/crash_reports/_mapping)' -d '{ "crash\_reports": { "date\_formats": [ "yyyy-MM-dd HH:mm:ss.SSSSSS" ], "properties": { "completeddatetime": { "format": "yyyy-MM-dd HH:mm:ss.SSSSSS", "type": "date" }, "date\_processed": { "format": "yyyy-MM-dd HH:mm:ss.SSSSSS", "type": "date" }, "client\_crash\_date": { "format": "yyyy-MM-dd HH:mm:ss.SSSSSS", "type": "date" }, "build\_date": { "format": "yyyy-MM-dd HH:mm:ss.SSSSSS", "type": "date" }, "startedDateTime": { "format": "yyyy-MM-dd HH:mm:ss.SSSSSS", "type": "date" }, "signature": { "type": "multi\_field", "fields": { "signature": { "type": "string", "index": "analyzed" }, "full": { "type": "string", "index": "not\_analyzed" } } } } } }' No OOME are thrown if the custom mappings aren't added to the index. Any idea what might be going wrong here? -anurag ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [July 5, 2011, 11:00am UTC](https://discuss.elastic.co/t/oome-when-using-mapping/4775/2 "2011-07-05T11:00:19Z")
Do you get the OOM right when you start to index, or after a time? Is there a chance that you can provide a link for me to download the heap dump generated? On Tuesday, July 5, 2011 at 7:12 AM, Anurag wrote: > We are running a 5-node Elasticsearch 0.16.2 cluster, avg. size 20kb / > document. However, I get an OutOfMemoryException on trying to insert > 1million JSONS with the following mapping. > > curl -XPUT '[http://localhost:9200/socorro/](http://localhost:9200/socorro/)' -d ' > index : > number\_of\_shards : 30 > number\_of\_replicas : 2 > ' > > curl -XPUT '[http://localhost:9200/socorro/crash\_reports/\_mapping](http://localhost:9200/socorro/crash_reports/_mapping)' -d '{ > "crash\_reports": { > "date\_formats": [ > "yyyy-MM-dd HH:mm:ss.SSSSSS" > ], > "properties": { > "completeddatetime": { > "format": "yyyy-MM-dd HH:mm:ss.SSSSSS", > "type": "date" > }, > "date\_processed": { > "format": "yyyy-MM-dd HH:mm:ss.SSSSSS", > "type": "date" > }, > "client\_crash\_date": { > "format": "yyyy-MM-dd HH:mm:ss.SSSSSS", > "type": "date" > }, > "build\_date": { > "format": "yyyy-MM-dd HH:mm:ss.SSSSSS", > "type": "date" > }, > "startedDateTime": { > "format": "yyyy-MM-dd HH:mm:ss.SSSSSS", > "type": "date" > }, > "signature": { > "type": "multi\_field", > "fields": { > "signature": { > "type": "string", > "index": "analyzed" > }, > "full": { > "type": "string", > "index": "not\_analyzed" > } > } > } > } > } > }' > > No OOME are thrown if the custom mappings aren't added to the index. > Any idea what might be going wrong here? > > -anurag ---
**Author:** ![Anurag\_Phadke](https://avatars.discourse-cdn.com/v4/letter/a/5f9b8f/32.png) [@Anurag\_Phadke](https://discuss.elastic.co/u/Anurag_Phadke)\ **Post date:** [July 5, 2011, 5:00pm UTC](https://discuss.elastic.co/t/oome-when-using-mapping/4775/3 "2011-07-05T17:00:02Z")
Shay, I don't get the error when starting the index, it happens after inserting about 100k jsons. Here's the dump: java.lang.OutOfMemoryError: Java heap space java.lang.OutOfMemoryError: Java heap space java.lang.OutOfMemoryError: Java heap space Exception in thread "HashSessionScavenger-0" java.lang.OutOfMemoryError: Java heap space Exception in thread "New I/O client worker #1-4" java.lang.OutOfMemoryError: Java heap space Exception in thread "elasticsearch[Madame MacEvil]transport\_client\_worker-pool-13-thread-8" java.lang.OutOfMemoryError: Java heap space Exception in thread "elasticsearch[Madame MacEvil]discovery#multicast#received-pool-16-thread-1" java.lang.OutOfMemoryError: Java heap space Exception in thread "elasticsearch[Madame MacEvil]transport\_client\_worker-pool-13-thread-2" java.lang.OutOfMemoryError: Java heap space java.lang.OutOfMemoryError: Java heap space On Tue, Jul 5, 2011 at 4:00 AM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote: > Do you get the OOM right when you start to index, or after a time? Is there > a chance that you can provide a link for me to download the heap dump > generated? > > On Tuesday, July 5, 2011 at 7:12 AM, Anurag wrote: > > We are running a 5-node Elasticsearch 0.16.2 cluster, avg. size 20kb / > document. However, I get an OutOfMemoryException on trying to insert > 1million JSONS with the following mapping. > > curl -XPUT '[http://localhost:9200/socorro/](http://localhost:9200/socorro/)' -d ' > index : > number\_of\_shards : 30 > number\_of\_replicas : 2 > ' > > curl -XPUT '[http://localhost:9200/socorro/crash\_reports/\_mapping](http://localhost:9200/socorro/crash_reports/_mapping)' -d '{ > "crash\_reports": { > "date\_formats": [ > "yyyy-MM-dd HH:mm:ss.SSSSSS" > ], > ""properties": { > "completeddatetime": { > ""format": "yyyy-MM-dd HH:mm:ss.SSSSSS", > ""type": "date" > }, > "date\_processed": { > "format": "yyyy-MM-dd HH:mm:ss.SSSSSS", > "type": "date" > }, > "client\_crash\_date": { > "format": "yyyy-MM-dd HH:mm:ss.SSSSSS", > "type": "date" > }, > "build\_date": { > "format": "yyyy-MM-dd HH:mm:ss.SSSSSS", > ""type": "date" > }, > "startedDateTime": { > "format": "yyyy-MM-dd HH:mm:ss.SSSSSS", > "type": "date" > }}, > "signature": { > "type": "multi\_field", > "fields": { > ""signature": { > "type": "string", > "index": "analyzed" > }}, > "full": { > ""type": "string", > "index": "not\_analyzed" > } > } > } > } > } > }' > > No OOME are thrown if the custom mappings aren't added to the index. > Any idea what might be going wrong here? > > -anurag ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [July 6, 2011, 1:24am UTC](https://discuss.elastic.co/t/oome-when-using-mapping/4775/4 "2011-07-06T01:24:38Z")
Thats the exceptions, not the heap dump... . How are you running the indexing? Is it against an embedded instance? Do you have enough memory? Are you using bulk indexing, if so, whats your bulk size? On Tuesday, July 5, 2011 at 8:00 PM, Anurag wrote: > Shay, > I don't get the error when starting the index, it happens after > inserting about 100k jsons. > Here's the dump: > java.lang.OutOfMemoryError: Java heap space > java.lang.OutOfMemoryError: Java heap space > java.lang.OutOfMemoryError: Java heap space > Exception in thread "HashSessionScavenger-0" > java.lang.OutOfMemoryError: Java heap space > Exception in thread "New I/O client worker #1-4" > java.lang.OutOfMemoryError: Java heap space > Exception in thread "elasticsearch[Madame > MacEvil]transport\_client\_worker-pool-13-thread-8" > java.lang.OutOfMemoryError: Java heap space > Exception in thread "elasticsearch[Madame > MacEvil]discovery#multicast#received-pool-16-thread-1" > java.lang.OutOfMemoryError: Java heap space > Exception in thread "elasticsearch[Madame > MacEvil]transport\_client\_worker-pool-13-thread-2" > java.lang.OutOfMemoryError: Java heap space > java.lang.OutOfMemoryError: Java heap space > > On Tue, Jul 5, 2011 at 4:00 AM, Shay Banon \<[shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) ([mailto:shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com))\> wrote: > > > Do you get the OOM right when you start to index, or after a time? Is there > > a chance that you can provide a link for me to download the heap dump > > generated? > > > > On Tuesday, July 5, 2011 at 7:12 AM, Anurag wrote: > > > > We are running a 5-node Elasticsearch 0.16.2 cluster, avg. size 20kb / > > document. However, I get an OutOfMemoryException on trying to insert > > 1million JSONS with the following mapping. > > > > curl -XPUT '[http://localhost:9200/socorro/](http://localhost:9200/socorro/)' -d ' > > index : > > number\_of\_shards : 30 > > number\_of\_replicas : 2 > > ' > > > > curl -XPUT '[http://localhost:9200/socorro/crash\_reports/\_mapping](http://localhost:9200/socorro/crash_reports/_mapping)' -d '{ > > "crash\_reports": { > > "date\_formats": [ > > "yyyy-MM-dd HH:mm:ss.SSSSSS" > > ], > > ""properties": { > > "completeddatetime": { > > ""format": "yyyy-MM-dd HH:mm:ss.SSSSSS", > > ""type": "date" > > }, > > "date\_processed": { > > "format": "yyyy-MM-dd HH:mm:ss.SSSSSS", > > "type": "date" > > }, > > "client\_crash\_date": { > > "format": "yyyy-MM-dd HH:mm:ss.SSSSSS", > > "type": "date" > > }, > > "build\_date": { > > "format": "yyyy-MM-dd HH:mm:ss.SSSSSS", > > ""type": "date" > > }, > > "startedDateTime": { > > "format": "yyyy-MM-dd HH:mm:ss.SSSSSS", > > "type": "date" > > }}, > > "signature": { > > "type": "multi\_field", > > "fields": { > > ""signature": { > > "type": "string", > > "index": "analyzed" > > }}, > > "full": { > > ""type": "string", > > "index": "not\_analyzed" > > } > > } > > } > > } > > } > > }' > > > > No OOME are thrown if the custom mappings aren't added to the index. > > Any idea what might be going wrong here? > > > > -anurag ---
**Author:** ![Anurag\_Phadke](https://avatars.discourse-cdn.com/v4/letter/a/5f9b8f/32.png) [@Anurag\_Phadke](https://discuss.elastic.co/u/Anurag_Phadke)\ **Post date:** [July 6, 2011, 4:31am UTC](https://discuss.elastic.co/t/oome-when-using-mapping/4775/5 "2011-07-06T04:31:56Z")
Will try to get u the heap dump by tomorrow Indexing is against an embedded instance, xms as 2g and Xmx as 8g. Using bulk indexing with 100 jsons, each approximately 20kb in size. On Jul 5, 2011, at 6:24 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote: > Thats the exceptions, not the heap dump... . How are you running the indexing? Is it against an embedded instance? Do you have enough memory? Are you using bulk indexing, if so, whats your bulk size? > On Tuesday, July 5, 2011 at 8:00 PM, Anurag wrote: > > > Shay, > > I don't get the error when starting the index, it happens after > > inserting about 100k jsons. > > Here's the dump: > > java.lang.OutOfMemoryError: Java heap space > > java.lang.OutOfMemoryError: Java heap space > > java.lang.OutOfMemoryError: Java heap space > > Exception in thread "HashSessionScavenger-0" > > java.lang.OutOfMemoryError: Java heap space > > Exception in thread "New I/O client worker #1-4" > > java.lang.OutOfMemoryError: Java heap space > > Exception in thread "elasticsearch[Madame > > MacEvil]transport\_client\_worker-pool-13-thread-8" > > java.lang.OutOfMemoryError: Java heap space > > Exception in thread "elasticsearch[Madame > > MacEvil]discovery#multicast#received-pool-16-thread-1" > > java.lang.OutOfMemoryError: Java heap space > > Exception in thread "elasticsearch[Madame > > MacEvil]transport\_client\_worker-pool-13-thread-2" > > java.lang.OutOfMemoryError: Java heap space > > java.lang.OutOfMemoryError: Java heap space > > > > On Tue, Jul 5, 2011 at 4:00 AM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote: > > > > > Do you get the OOM right when you start to index, or after a time? Is there > > > a chance that you can provide a link for me to download the heap dump > > > generated? > > > > > > On Tuesday, July 5, 2011 at 7:12 AM, Anurag wrote: > > > > > > We are running a 5-node Elasticsearch 0.16.2 cluster, avg. size 20kb / > > > document. However, I get an OutOfMemoryException on trying to insert > > > 1million JSONS with the following mapping. > > > > > > curl -XPUT '[http://localhost:9200/socorro/](http://localhost:9200/socorro/)' -d ' > > > index : > > > number\_of\_shards : 30 > > > number\_of\_replicas : 2 > > > ' > > > > > > curl -XPUT '[http://localhost:9200/socorro/crash\_reports/\_mapping](http://localhost:9200/socorro/crash_reports/_mapping)' -d '{ > > > "crash\_reports": { > > > "date\_formats": [ > > > "yyyy-MM-dd HH:mm:ss.SSSSSS" > > > ], > > > ""properties": { > > > "completeddatetime": { > > > ""format": "yyyy-MM-dd HH:mm:ss.SSSSSS", > > > ""type": "date" > > > }, > > > "date\_processed": { > > > "format": "yyyy-MM-dd HH:mm:ss.SSSSSS", > > > "type": "date" > > > }, > > > "client\_crash\_date": { > > > "format": "yyyy-MM-dd HH:mm:ss.SSSSSS", > > > "type": "date" > > > }, > > > "build\_date": { > > > "format": "yyyy-MM-dd HH:mm:ss.SSSSSS", > > > ""type": "date" > > > }, > > > "startedDateTime": { > > > "format": "yyyy-MM-dd HH:mm:ss.SSSSSS", > > > "type": "date" > > > }}, > > > "signature": { > > > "type": "multi\_field", > > > "fields": { > > > ""signature": { > > > "type": "string", > > > "index": "analyzed" > > > }}, > > > "full": { > > > ""type": "string", > > > "index": "not\_analyzed" > > > } > > > } > > > } > > > } > > > } > > > }' > > > > > > No OOME are thrown if the custom mappings aren't added to the index. > > > Any idea what might be going wrong here? > > > > > > -anurag ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:01am UTC](https://discuss.elastic.co/t/oome-when-using-mapping/4775/6 "2017-07-06T04:01:39Z")