ES query is a little bit slow, can anyone help have a look?

Elastic Stack Elasticsearch
1

The messages from "get _nodes/hot_threads" via kibana dev tools.

::: {testdata07.tls.ad}{xWqVDOy9SpWhs9QrxWygaA}{Lj7vbnrnRHaW-MSqc62PVA}{192.168.0.127}{192.168.0.127:9300}
   Hot threads at 2017-11-10T04:10:45.706Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:

::: {testdata12.tls.ad}{SzWp_JwIQNSXXgcUpRhz5Q}{ZJ06ULCiRB6WnKaGWWe3PA}{192.168.0.132}{192.168.0.132:9300}
   Hot threads at 2017-11-10T04:10:45.690Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:
   
    2.2% (11ms out of 500ms) cpu usage by thread 'elasticsearch[testdata12.tls.ad][[transport_server_worker.default]][T#45]'
     7/10 snapshots sharing following 2 elements
       io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
       java.lang.Thread.run(Thread.java:748)

::: {testdata06.tls.ad}{0jL7TbdMQWqur6WGL6POrQ}{YlwTUeKxSG2aqT9nIJ8wQw}{192.168.0.126}{192.168.0.126:9300}
   Hot threads at 2017-11-10T04:10:45.697Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:

::: {testdata03.tls.ad}{ucAHPSkdQRuuZZytehLs1w}{5dp-SRXmRr2fRSUdF0D_IA}{192.168.0.123}{192.168.0.123:9300}
   Hot threads at 2017-11-10T04:10:45.717Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:
   
   106.0% (529.9ms out of 500ms) cpu usage by thread 'elasticsearch[testdata03.tls.ad][[metricbeat-2017.45][1]: Lucene Merge Thread #194]'
     2/10 snapshots sharing following 24 elements
       org.apache.lucene.util.LongValues.get(LongValues.java:45)
       org.apache.lucene.index.SingletonSortedNumericDocValues.setDocument(SingletonSortedNumericDocValues.java:52)
       org.apache.lucene.codecs.DocValuesConsumer$SortedNumericDocValuesSub.nextDoc(DocValuesConsumer.java:449)
       org.apache.lucene.index.DocIDMerger$SequentialDocIDMerger.next(DocIDMerger.java:100)
       org.apache.lucene.codecs.DocValuesConsumer$4$1.setNext(DocValuesConsumer.java:573)
       org.apache.lucene.codecs.DocValuesConsumer$4$1.hasNext(DocValuesConsumer.java:545)
       org.apache.lucene.codecs.DocValuesConsumer$4$1.next(DocValuesConsumer.java:555)
       org.apache.lucene.codecs.DocValuesConsumer$4$1.next(DocValuesConsumer.java:536)
       org.apache.lucene.codecs.DocValuesConsumer$10$1.next(DocValuesConsumer.java:1028)
       org.apache.lucene.codecs.DocValuesConsumer$10$1.next(DocValuesConsumer.java:1015)
       org.apache.lucene.codecs.lucene54.Lucene54DocValuesConsumer.writeSparseMissingBitset(Lucene54DocValuesConsumer.java:332)
       org.apache.lucene.codecs.lucene54.Lucene54DocValuesConsumer.addNumericField(Lucene54DocValuesConsumer.java:207)
       org.apache.lucene.codecs.lucene54.Lucene54DocValuesConsumer.addNumericField(Lucene54DocValuesConsumer.java:89)
       org.apache.lucene.codecs.lucene54.Lucene54DocValuesConsumer.addSortedNumericField(Lucene54DocValuesConsumer.java:589)
       org.apache.lucene.codecs.DocValuesConsumer.mergeSortedNumericField(DocValuesConsumer.java:470)
       org.apache.lucene.codecs.DocValuesConsumer.merge(DocValuesConsumer.java:243)
       org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.merge(PerFieldDocValuesFormat.java:153)
       org.apache.lucene.index.SegmentMerger.mergeDocValues(SegmentMerger.java:167)
       org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:111)
       org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4356)
       org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3931)
       org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:624)
       org.elasticsearch.index.engine.ElasticsearchConcurrentMergeScheduler.doMerge(ElasticsearchConcurrentMergeScheduler.java:99)
       org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:661)
     2/10 snapshots sharing following 15 elements
       org.apache.lucene.codecs.DocValuesConsumer$10$1.next(DocValuesConsumer.java:1015)
       org.apache.lucene.codecs.lucene54.Lucene54DocValuesConsumer.writeSparseMissingBitset(Lucene54DocValuesConsumer.java:332)
       org.apache.lucene.codecs.lucene54.Lucene54DocValuesConsumer.addNumericField(Lucene54DocValuesConsumer.java:207)
       org.apache.lucene.codecs.lucene54.Lucene54DocValuesConsumer.addNumericField(Lucene54DocValuesConsumer.java:89)
       org.apache.lucene.codecs.lucene54.Lucene54DocValuesConsumer.addSortedNumericField(Lucene54DocValuesConsumer.java:589)
       org.apache.lucene.codecs.DocValuesConsumer.mergeSortedNumericField(DocValuesConsumer.java:470)
       org.apache.lucene.codecs.DocValuesConsumer.merge(DocValuesConsumer.java:243)
2

Hey,

could you also write up a question instead of just posting a snippet? It's very hard to know, what your intention is?

The hot_threads output contains information, that elasticsearch is currently busy with merging - which is nothing bad, as it is doing some housekeeping on your indexed data. At some point merging will be done (unless you keep indexing data, then at some point in time there will be merging) and CPU will go down again.

--Alex

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.