# High memory usage with large number of fields **URL:** \ **Category:** Elasticsearch\ **Created:** [November 17, 2014, 6:41pm UTC](https://discuss.elastic.co/t/high-memory-usage-with-large-number-of-fields/20791 "2014-11-17T18:41:55Z")\ **Posts on this page:** 2\ **Page:** 1
**Author:** ![Henrik\_Nordvik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/henrik_nordvik/32/1096_2.png) [@Henrik\_Nordvik](https://discuss.elastic.co/u/Henrik_Nordvik)\ **Post date:** [November 17, 2014, 6:41pm UTC](https://discuss.elastic.co/t/high-memory-usage-with-large-number-of-fields/20791/1 "2014-11-17T18:41:55Z")
Hi, I'm currently experiencing high memory usage and constant GC, and I'm trying to figure out why, and what I can do with it. I'm running on 1 node, ES 1.3.4, 6GB heap. Dataset on disk is 7.8GB, around 600.000 documents. This is mostly time series data, that is, a lot of numeric values that is faceted on, plus some not\_analyized strings for filtering. We use one index per day to make deleting old data fast, with one shard. The mapping contains about 3700 different fields per day, currently 75000 in total. I have tried to enable docvalues, disable \_source, disable \_all and set everything to not\_analyzed to reduce memory footprint, but it didn't help in this case. Are there any other knobs and switches I can tune to reduce memory usage? Are we just using too many fields? We chose this format because 1. it makes it easy to create graphs in kibana with facets on the different fields. 2. Simiar fields contain a lot of similar values, thus hopefully getting better compression than a "flatter"/more homogeneous mapping. As you see from the heap dumps there's a about 1 million instances of FST, FieldReader, FieldInfo, etc. My understanding was that FST was used for the terms dictionary, and we only have numeric and not\_analyzed fields. Are there any features I can disable? Heap histogram from jmap after outofmemory: ## num #instances#bytes Class description 1: 985378 520274216 org.apache.lucene.util.fst.FST$Arc[] 2: 7448423 238349536 java.util.HashMap$Entry 3: 9436225 226469400 java.lang.String 4: 5172693 184341744 byte[] 5: 3942232 157689280 java.util.TreeMap$Entry 6: 997087 106762648 java.util.HashMap$Entry[] 7: 985367 102478168 org.apache.lucene.util.fst.FST 8: 4158990 99815760 org.apache.lucene.util.BytesRef 9: 985366 86712208 org.apache.lucene.codecs.blocktree.FieldReader 10: 983946 70844112 org.apache.lucene.codecs.lucene49.Lucene49DocValuesProducer$BinaryEntry 11: 1014191 62432544 java.lang.Object[] 12: 1005699 48273552 java.util.HashMap 13: 985721 47314608 org.apache.lucene.index.FieldInfo 14: 136955 40463928 long[] 15: 985367 39414680 org.apache.lucene.util.fst.BytesStore 16: 2232203 35715248 java.lang.Integer 17: 986282 31561024 java.util.Collections$UnmodifiableMap 18: 397909 28649448 org.apache.lucene.util.fst.FST$Arc 19: 582434 27956832 org.elasticsearch.common.cache.LocalCache$StrongAccessEntry 20: 991896 23805504 java.util.ArrayList 21: 532630 17044160 org.elasticsearch.common.util.concurrent.jsr166e.ConcurrentHashMapV8$Node 22: 988834 16344944 int[] 23: 582302 13975248 org.elasticsearch.index.cache.filter.weighted.WeightedFilterCache$FilterCacheKey 24: 582302 13975248 org.elasticsearch.common.cache.LocalCache$WeightedStrongValueReference Heap histogram after restarting the application: ## num #instances#bytes class name 1: 9735025 758087288 [C 2: 1020815 538987880 [Lorg.apache.lucene.util.fst.FST$Arc; 3: 7710294 246729408 java.util.HashMap$Entry 4: 9727788 233466912 java.lang.String 5: 5351244 186234680 [B 6: 4083984 163359360 java.util.TreeMap$Entry 7: 1032416 110492320 [Ljava.util.HashMap$Entry; 8: 1020810 106164240 org.apache.lucene.util.fst.FST 9: 4304247 103301928 org.apache.lucene.util.BytesRef 10: 1020809 89831192 org.apache.lucene.codecs.blocktree.FieldReader 11: 1019357 73393704 org.apache.lucene.codecs.lucene49.Lucene49DocValuesProducer$BinaryEntry 12: 1051189 61103176 [Ljava.lang.Object; 13: 1040457 49941936 java.util.HashMap 14: 1021172 49016256 org.apache.lucene.index.FieldInfo 15: 1020810 40832400 org.apache.lucene.util.fst.BytesStore 16: 2312797 37004752 java.lang.Integer 17: 1021696 32694272 java.util.Collections$UnmodifiableMap 18: 413671 29784312 org.apache.lucene.util.fst.FST$Arc 19: 1029315 24703560 java.util.ArrayList 20: 1024198 16883184 [I - Henrik Nordvik -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/379ae367-e0d1-4e66-bbab-022bf3a9a9ed%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/379ae367-e0d1-4e66-bbab-022bf3a9a9ed%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:49am UTC](https://discuss.elastic.co/t/high-memory-usage-with-large-number-of-fields/20791/2 "2017-07-06T00:49:28Z")