Hi,
here is the result of hot threas api
::: [AEPLES2][joZ3M1MBStqL8HPiiNGbjg][AEPLPERF2][inet[/192.168.1.14:9300]]
9.4% (46.8ms out of 500ms) cpu usage by thread 'elasticsearch[AEPLES2][refresh][T#2]'
10/10 snapshots sharing following 9 elements
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:734)
java.util.concurrent.LinkedTransferQueue.xfer(LinkedTransferQueue.java:647)
java.util.concurrent.LinkedTransferQueue.poll(LinkedTransferQueue.java:1273)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
6.2% (31.2ms out of 500ms) cpu usage by thread 'elasticsearch[AEPLES2][search][T#2]'
10/10 snapshots sharing following 10 elements
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:737)
java.util.concurrent.LinkedTransferQueue.xfer(LinkedTransferQueue.java:647)
java.util.concurrent.LinkedTransferQueue.take(LinkedTransferQueue.java:1265)
org.elasticsearch.common.util.concurrent.SizeBlockingQueue.take(SizeBlockingQueue.java:162)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
6.2% (31.2ms out of 500ms) cpu usage by thread 'elasticsearch[AEPLES2][search][T#6]'
10/10 snapshots sharing following 10 elements
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:737)
java.util.concurrent.LinkedTransferQueue.xfer(LinkedTransferQueue.java:647)
java.util.concurrent.LinkedTransferQueue.take(LinkedTransferQueue.java:1265)
org.elasticsearch.common.util.concurrent.SizeBlockingQueue.take(SizeBlockingQueue.java:162)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
::: [AEPLES1][5D0X10n-RbWT3jU7k6lGog][AEPLPERF1][inet[/192.168.1.13:9300]]
3.1% (15.6ms out of 500ms) cpu usage by thread 'elasticsearch[AEPLES1][search][T#2]'
10/10 snapshots sharing following 10 elements
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:737)
java.util.concurrent.LinkedTransferQueue.xfer(LinkedTransferQueue.java:647)
java.util.concurrent.LinkedTransferQueue.take(LinkedTransferQueue.java:1265)
org.elasticsearch.common.util.concurrent.SizeBlockingQueue.take(SizeBlockingQueue.java:162)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
3.1% (15.6ms out of 500ms) cpu usage by thread 'elasticsearch[AEPLES1][search][T#7]'
9/10 snapshots sharing following 10 elements
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:737)
java.util.concurrent.LinkedTransferQueue.xfer(LinkedTransferQueue.java:647)
java.util.concurrent.LinkedTransferQueue.take(LinkedTransferQueue.java:1265)
org.elasticsearch.common.util.concurrent.SizeBlockingQueue.take(SizeBlockingQueue.java:162)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
unique snapshot
org.elasticsearch.common.lucene.search.XBooleanFilter.getDocIdSet(XBooleanFilter.java:292)
org.elasticsearch.search.aggregations.bucket.filter.FilterAggregator.setNextReader(FilterAggregator.java:52)
org.elasticsearch.search.aggregations.support.AggregationContext.setNextReader(AggregationContext.java:94)
org.elasticsearch.search.aggregations.AggregationPhase$AggregationsCollector.setNextReader(AggregationPhase.java:167)
org.elasticsearch.common.lucene.MultiCollector.setNextReader(MultiCollector.java:68)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:612)
org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:191)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:491)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:448)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:281)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:269)
org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:157)
org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:272)
org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:283)
org.elasticsearch.search.action.SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:231)
org.elasticsearch.search.action.SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:228)
org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:559)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
3.1% (15.6ms out of 500ms) cpu usage by thread 'elasticsearch[AEPLES1][transport_client_worker][T#4]{New I/O worker #4}'
10/10 snapshots sharing following 16 elements
sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
org.elasticsearch.common.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:68)
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.select(AbstractNioSelector.java:415)
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:212)
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
On Wednesday, May 6, 2015 at 2:11:20 PM UTC+5:30, David Pilato wrote:
May be running hot_threads API would tell you more?
BTW which version is it ?
--
David 
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
Le 6 mai 2015 à 10:33, Chetan Dev <chete...@carwale.com <javascript:>> a
écrit :
Hi,
Servers were at 20% cpu but after sudden restart java started taking more
cpu currently cpu utilization is 90%
I digged into the logs and found this
at
org.elasticsearch.search.SearchService.parseSource(SearchService.java:687)
at
org.elasticsearch.search.SearchService.createContext(SearchService.java:543)
at
org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:515)
at
org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:277)
at
org.elasticsearch.search.action.SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:231)
at
org.elasticsearch.search.action.SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:228)
at
org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:559)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NumberFormatException: For input string:
"${map_makes:${lowercase:}}"
at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Long.parseLong(Long.java:578)
at java.lang.Long.parseLong(Long.java:631)
at
org.elasticsearch.index.mapper.core.NumberFieldMapper.parseLongValue(NumberFieldMapper.java:354)
at
org.elasticsearch.index.mapper.core.LongFieldMapper.indexedValueForSearch(LongFieldMapper.java:169)
at
org.elasticsearch.index.mapper.core.AbstractFieldMapper.termsFilter(AbstractFieldMapper.java:483)
at
org.elasticsearch.index.query.TermsFilterParser.parse(TermsFilterParser.java:201)
at
org.elasticsearch.index.query.QueryParseContext.executeFilterParser(QueryParseContext.java:347)
at
org.elasticsearch.index.query.QueryParseContext.parseInnerFilter(QueryParseContext.java:328)
at
org.elasticsearch.index.query.BoolFilterParser.parse(BoolFilterParser.java:92)
at
org.elasticsearch.index.query.QueryParseContext.executeFilterParser(QueryParseContext.java:347)
at
org.elasticsearch.index.query.QueryParseContext.parseInnerFilter(QueryParseContext.java:328)
at
org.elasticsearch.index.query.BoolFilterParser.parse(BoolFilterParser.java:92)
at
org.elasticsearch.index.query.QueryParseContext.executeFilterParser(QueryParseContext.java:347)
at
org.elasticsearch.index.query.QueryParseContext.parseInnerFilter(QueryParseContext.java:328)
at
org.elasticsearch.index.query.IndexQueryParserService.parseInnerFilter(IndexQueryParserService.java:295)
at
org.elasticsearch.search.aggregations.bucket.filter.FilterParser.parse(FilterParser.java:42)
at
org.elasticsearch.search.aggregations.AggregatorParsers.parseAggregators(AggregatorParsers.java:130)
at
org.elasticsearch.search.aggregations.AggregatorParsers.parseAggregators(AggregatorParsers.java:77)
at
org.elasticsearch.search.aggregations.AggregationParseElement.parse(AggregationParseElement.java:60)
at
org.elasticsearch.search.SearchService.parseSource(SearchService.java:671)
... 9 more
[2015-05-06 12:27:16,506][DEBUG][action.search.type ] [AEPLES1] All
shards failed for phase: [query]
On Wednesday, May 6, 2015 at 1:15:27 PM UTC+5:30, David Pilato wrote:
What is wrong ?
I mean that if you are doing indexing for example, that's fine if your
CPU is used.
But may be you are running out of memory and GC is running all the time ?
Hard to say without any details / logs...
BTW me moved to https://discuss.elastic.co/
--
David
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
Le 6 mai 2015 à 09:28, Chetan Dev chete...@carwale.com a écrit :
Hi,
I am running elasticsearch 1.4.4 with java 1.8.0.25 on windows machine .
i have assigned 2 Gb heap memory
java.exe taking almost 70 per cpu ?
everything else is working fine
Can somebody please give right direction to diagnose what is causing this
fluctuation
Thanks
--
Please update your bookmarks! We moved to https://discuss.elastic.co/
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/c2860555-c430-4301-b7ec-e25d4187bac6%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/c2860555-c430-4301-b7ec-e25d4187bac6%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.
--
Please update your bookmarks! We moved to https://discuss.elastic.co/
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/186606d3-bd65-4325-bd15-090291ab8694%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/186606d3-bd65-4325-bd15-090291ab8694%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e2af438e-9413-4a47-b546-8efa671224ec%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.