Hi,
I'm currently experiencing high memory usage and constant GC, and I'm
trying to figure out why, and what I can do with it.
I'm running on 1 node, ES 1.3.4, 6GB heap. Dataset on disk is 7.8GB, around
600.000 documents.
This is mostly time series data, that is, a lot of numeric values that is
faceted on, plus some not_analyized strings for filtering.
We use one index per day to make deleting old data fast, with one shard.
The mapping contains about 3700 different fields per day, currently 75000
in total.
I have tried to enable docvalues, disable _source, disable _all and set
everything to not_analyzed to reduce memory footprint, but it didn't help
in this case.
Are there any other knobs and switches I can tune to reduce memory usage?
Are we just using too many fields? We chose this format because
- it makes it easy to create graphs in kibana with facets on the different
fields. - Simiar fields contain a lot of similar values, thus hopefully getting
better compression than a "flatter"/more homogeneous mapping.
As you see from the heap dumps there's a about 1 million instances of FST,
FieldReader, FieldInfo, etc.
My understanding was that FST was used for the terms dictionary, and we
only have numeric and not_analyzed fields. Are there any features I can
disable?
Heap histogram from jmap after outofmemory:
num #instances #bytes Class description
1: 985378 520274216 org.apache.lucene.util.fst.FST$Arc[]
2: 7448423 238349536 java.util.HashMap$Entry
3: 9436225 226469400 java.lang.String
4: 5172693 184341744 byte[]
5: 3942232 157689280 java.util.TreeMap$Entry
6: 997087 106762648 java.util.HashMap$Entry[]
7: 985367 102478168 org.apache.lucene.util.fst.FST
8: 4158990 99815760 org.apache.lucene.util.BytesRef
9: 985366 86712208
org.apache.lucene.codecs.blocktree.FieldReader
10: 983946 70844112
org.apache.lucene.codecs.lucene49.Lucene49DocValuesProducer$BinaryEntry
11: 1014191 62432544 java.lang.Object[]
12: 1005699 48273552 java.util.HashMap
13: 985721 47314608 org.apache.lucene.index.FieldInfo
14: 136955 40463928 long[]
15: 985367 39414680
org.apache.lucene.util.fst.BytesStore
16: 2232203 35715248 java.lang.Integer
17: 986282 31561024
java.util.Collections$UnmodifiableMap
18: 397909 28649448 org.apache.lucene.util.fst.FST$Arc
19: 582434 27956832
org.elasticsearch.common.cache.LocalCache$StrongAccessEntry
20: 991896 23805504 java.util.ArrayList
21: 532630 17044160
org.elasticsearch.common.util.concurrent.jsr166e.ConcurrentHashMapV8$Node
22: 988834 16344944 int[]
23: 582302 13975248
org.elasticsearch.index.cache.filter.weighted.WeightedFilterCache$FilterCacheKey
24: 582302 13975248
org.elasticsearch.common.cache.LocalCache$WeightedStrongValueReference
Heap histogram after restarting the application:
num #instances #bytes class name
1: 9735025 758087288 [C
2: 1020815 538987880 [Lorg.apache.lucene.util.fst.FST$Arc;
3: 7710294 246729408 java.util.HashMap$Entry
4: 9727788 233466912 java.lang.String
5: 5351244 186234680 [B
6: 4083984 163359360 java.util.TreeMap$Entry
7: 1032416 110492320 [Ljava.util.HashMap$Entry;
8: 1020810 106164240 org.apache.lucene.util.fst.FST
9: 4304247 103301928 org.apache.lucene.util.BytesRef
10: 1020809 89831192
org.apache.lucene.codecs.blocktree.FieldReader
11: 1019357 73393704
org.apache.lucene.codecs.lucene49.Lucene49DocValuesProducer$BinaryEntry
12: 1051189 61103176 [Ljava.lang.Object;
13: 1040457 49941936 java.util.HashMap
14: 1021172 49016256 org.apache.lucene.index.FieldInfo
15: 1020810 40832400 org.apache.lucene.util.fst.BytesStore
16: 2312797 37004752 java.lang.Integer
17: 1021696 32694272 java.util.Collections$UnmodifiableMap
18: 413671 29784312 org.apache.lucene.util.fst.FST$Arc
19: 1029315 24703560 java.util.ArrayList
20: 1024198 16883184 [I
Henrik Nordvik
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/379ae367-e0d1-4e66-bbab-022bf3a9a9ed%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.