# Elasticsearch performance issue with load increasing reponse time is increasing **URL:** \ **Category:** Elasticsearch\ **Created:** [February 27, 2014, 9:32am UTC](https://discuss.elastic.co/t/elasticsearch-performance-issue-with-load-increasing-reponse-time-is-increasing/16049 "2014-02-27T09:32:29Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![kondapallinaresh](https://avatars.discourse-cdn.com/v4/letter/k/dec6dc/32.png) [@kondapallinaresh](https://discuss.elastic.co/u/kondapallinaresh)\ **Post date:** [February 27, 2014, 9:32am UTC](https://discuss.elastic.co/t/elasticsearch-performance-issue-with-load-increasing-reponse-time-is-increasing/16049/1 "2014-02-27T09:32:29Z")
Hi All We had elasticsearch with 4 Nodes the the Cluster each 8gb memory. when we are calling the elastic search query from java with 25 threads first thread is taking 400 msec where as last is taking around 5secs its more than 10 times. we had 50 million of data with 5 shards and 0 replicas. we are monitoring the jvm everything is running less than 25% usage and cpu usage too is fine. do we need to have any setting in yml to improve the performance. here are logs for response time. The time taken to retrun data serial number search::470 The time taken to retrun data serial number search::471 The time taken to retrun data serial number search::490 The time taken to retrun data serial number search::491 The time taken to retrun data serial number search::501 The time taken to retrun data serial number search::502 The time taken to retrun data serial number search::440 The time taken to retrun data serial number search::441 The time taken to retrun data serial number search::489 The time taken to retrun data serial number search::490 The time taken to retrun data serial number search::461 The time taken to retrun data serial number search::462 The time taken to retrun data serial number search::497 The time taken to retrun data serial number search::498 The time taken to retrun data serial number search::534 The time taken to retrun data serial number search::510 The time taken to retrun data serial number search::535 The time taken to retrun data serial number search::511 The time taken to retrun data serial number search::524 The time taken to retrun data serial number search::525 The time taken to retrun data serial number search::488 The time taken to retrun data serial number search::489 The time taken to retrun data serial number search::602 The time taken to retrun data serial number search::603 The time taken to retrun data serial number search::627 The time taken to retrun data serial number search::628 The time taken to retrun data serial number search::613 The time taken to retrun data serial number search::614 The time taken to retrun data serial number search::668 The time taken to retrun data serial number search::669 The time taken to retrun data serial number search::702 The time taken to retrun data serial number search::704 The time taken to retrun data serial number search::717 The time taken to retrun data serial number search::718 The time taken to retrun data serial number search::689 The time taken to retrun data serial number search::689 The time taken to retrun data serial number search::742 The time taken to retrun data serial number search::743 The time taken to retrun data serial number search::722 The time taken to retrun data serial number search::723 The time taken to retrun data serial number search::1005 The time taken to retrun data serial number search::1018 The time taken to retrun data serial number search::1006 The time taken to retrun data serial number search::1019 The time taken to retrun data serial number search::985 The time taken to retrun data serial number search::986 The time taken to retrun data serial number search::1078 The time taken to retrun data serial number search::1079 The time taken to retrun data serial number search::1563 The time taken to retrun data serial number search::1564 The time taken to retrun data serial number search::1585 The time taken to retrun data serial number search::1586 The time taken to retrun data serial number search::1712 The time taken to retrun data serial number search::1713 The time taken to retrun data serial number search::2047 The time taken to retrun data serial number search::2048 The time taken to retrun data serial number search::2073 The time taken to retrun data serial number search::2074 The time taken to retrun data serial number search::2065 The time taken to retrun data serial number search::2066 The time taken to retrun data serial number search::2875 The time taken to retrun data serial number search::2876 The time taken to retrun data serial number search::3065 The time taken to retrun data serial number search::3065 The time taken to retrun data serial number search::3068 The time taken to retrun data serial number search::3069 The time taken to retrun data serial number search::3070 The time taken to retrun data serial number search::3075 The time taken to retrun data serial number search::3071 The time taken to retrun data serial number search::3076 The time taken to retrun data serial number search::3092 The time taken to retrun data serial number search::3093 The time taken to retrun data serial number search::3529 The time taken to retrun data serial number search::3530 The time taken to retrun data serial number search::3570 The time taken to retrun data serial number search::3570 The time taken to retrun data serial number search::4124 The time taken to retrun data serial number search::4125 The time taken to retrun data serial number search::4152 The time taken to retrun data serial number search::4153 The time taken to retrun data serial number search::4153 The time taken to retrun data serial number search::4154 The time taken to retrun data serial number search::4149 The time taken to retrun data serial number search::4150 The time taken to retrun data serial number search::5165 The time taken to retrun data serial number search::5165 The time taken to retrun data serial number search::5134 The time taken to retrun data serial number search::5135 The time taken to retrun data serial number search::5146 The time taken to retrun data serial number search::5146 The time taken to retrun data serial number search::5291 The time taken to retrun data serial number search::5292 The time taken to retrun data serial number search::5290 The time taken to retrun data serial number search::5291 The time taken to retrun data serial number search::5639 The time taken to retrun data serial number search::5640 Thanks, Naresh.K -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/3ba6f043-e5a6-43e0-8755-1ad96b6de185%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/3ba6f043-e5a6-43e0-8755-1ad96b6de185%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Binh\_Ly\_2](https://avatars.discourse-cdn.com/v4/letter/b/d07c76/32.png) [@Binh\_Ly\_2](https://discuss.elastic.co/u/Binh_Ly_2)\ **Post date:** [February 27, 2014, 2:21pm UTC](https://discuss.elastic.co/t/elasticsearch-performance-issue-with-load-increasing-reponse-time-is-increasing/16049/2 "2014-02-27T14:21:56Z")
While all the threads are running, can you check the search thread pool stats: curl "localhost:9200/\_nodes/stats/thread\_pool?pretty" It is possible that 1 or more searches are in the queue so they would wait until the search threadpool has availability again. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/9d4bc226-b6f7-4077-8196-6932b0841dba%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/9d4bc226-b6f7-4077-8196-6932b0841dba%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [February 27, 2014, 4:38pm UTC](https://discuss.elastic.co/t/elasticsearch-performance-issue-with-load-increasing-reponse-time-is-increasing/16049/3 "2014-02-27T16:38:49Z")
It would help if you can post the query you use, and how the lines you send correspond to the number of threads, and how you fire off the queries in parallel. Jörg -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoELuRqhiiFO5HO-LGB0d78tre-mi2YGLvUJ9VBLmjwwJA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoELuRqhiiFO5HO-LGB0d78tre-mi2YGLvUJ9VBLmjwwJA%40mail.gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![kondapallinaresh](https://avatars.discourse-cdn.com/v4/letter/k/dec6dc/32.png) [@kondapallinaresh](https://discuss.elastic.co/u/kondapallinaresh)\ **Post date:** [February 27, 2014, 11:56pm UTC](https://discuss.elastic.co/t/elasticsearch-performance-issue-with-load-increasing-reponse-time-is-increasing/16049/4 "2014-02-27T23:56:08Z")
Here is the query's i am using first query returns the parent instance id's based on those parent instance is trigirreng the second query. SearchResponse response = elasticClient.prepareSearch("ibdataindex").setTypes("coveredibsa").setQuery(QueryBuilders.termsQuery("SERIAL\_NUMBER", serialNo)) .setSearchType(SearchType.QUERY\_AND\_FETCH).addField("PARENT\_INSTANCE\_ID").setFrom(0) .setSize(20000) .execute() .actionGet(); results = response.getHits().hits(); LOGGER.info("The time taken to retrun data serial number search::"+ (System.currentTimeMillis()-time1)); String instanceQueryString = ""; int i=0; for (SearchHit hit : results) { instanceQueryString =instanceQueryString+" "+hit.getFields().get("PARENT\_INSTANCE\_ID").getValue(); } List\\> hits = new ArrayList\\>(); //qb = QueryBuilders.queryString("PARENT\_INSTANCE\_ID:("+instanceQueryString+")"); long time2 = System.currentTimeMillis(); if(!instanceQueryString.equals("")){ response = elasticClient.prepareSearch("ibdataindex") .setTypes("coveredibsa").setQuery(QueryBuilders.termsQuery("PARENT\_INSTANCE\_ID",serialNo)) .setSearchType(SearchType.QUERY\_AND\_FETCH).addFields(coulns.split(",")) .setFrom(0).setSize(10000) .execute() .actionGet(); I am using spring task scheduler in which i am passing 250 serial nos as batch. On Thursday, February 27, 2014 8:38:49 AM UTC-8, Jörg Prante wrote: > It would help if you can post the query you use, and how the lines you > send correspond to the number of threads, and how you fire off the queries > in parallel. > > Jörg -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/85442edf-c452-4c04-92af-34c4fc5fa420%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/85442edf-c452-4c04-92af-34c4fc5fa420%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![webish](https://avatars.discourse-cdn.com/v4/letter/w/919ad9/32.png) [@webish](https://discuss.elastic.co/u/webish)\ **Post date:** [May 14, 2014, 4:13am UTC](https://discuss.elastic.co/t/elasticsearch-performance-issue-with-load-increasing-reponse-time-is-increasing/16049/5 "2014-05-14T04:13:01Z")
Did you ever solve this? On Thursday, February 27, 2014 4:32:29 AM UTC-5, [kondapal...@gmail.com](mailto:kondapal...@gmail.com) wrote: > Hi All > > We had elasticsearch with 4 Nodes the the Cluster each 8gb memory. > when we are calling the Elasticsearch query from java with 25 threads > first thread is taking 400 msec where as last is taking around 5secs its > more than 10 times. > > we had 50 million of data with 5 shards and 0 replicas. > > we are monitoring the jvm everything is running less than 25% usage and > cpu usage too is fine. > > do we need to have any setting in yml to improve the performance. > > here are logs for response time. > > The time taken to retrun data serial number search::470 > The time taken to retrun data serial number search::471 > The time taken to retrun data serial number search::490 > The time taken to retrun data serial number search::491 > The time taken to retrun data serial number search::501 > The time taken to retrun data serial number search::502 > The time taken to retrun data serial number search::440 > The time taken to retrun data serial number search::441 > The time taken to retrun data serial number search::489 > The time taken to retrun data serial number search::490 > The time taken to retrun data serial number search::461 > The time taken to retrun data serial number search::462 > The time taken to retrun data serial number search::497 > The time taken to retrun data serial number search::498 > The time taken to retrun data serial number search::534 > The time taken to retrun data serial number search::510 > The time taken to retrun data serial number search::535 > The time taken to retrun data serial number search::511 > The time taken to retrun data serial number search::524 > The time taken to retrun data serial number search::525 > The time taken to retrun data serial number search::488 > The time taken to retrun data serial number search::489 > The time taken to retrun data serial number search::602 > The time taken to retrun data serial number search::603 > The time taken to retrun data serial number search::627 > The time taken to retrun data serial number search::628 > The time taken to retrun data serial number search::613 > The time taken to retrun data serial number search::614 > The time taken to retrun data serial number search::668 > The time taken to retrun data serial number search::669 > The time taken to retrun data serial number search::702 > The time taken to retrun data serial number search::704 > The time taken to retrun data serial number search::717 > The time taken to retrun data serial number search::718 > The time taken to retrun data serial number search::689 > The time taken to retrun data serial number search::689 > The time taken to retrun data serial number search::742 > The time taken to retrun data serial number search::743 > The time taken to retrun data serial number search::722 > The time taken to retrun data serial number search::723 > The time taken to retrun data serial number search::1005 > The time taken to retrun data serial number search::1018 > The time taken to retrun data serial number search::1006 > The time taken to retrun data serial number search::1019 > The time taken to retrun data serial number search::985 > The time taken to retrun data serial number search::986 > The time taken to retrun data serial number search::1078 > The time taken to retrun data serial number search::1079 > The time taken to retrun data serial number search::1563 > The time taken to retrun data serial number search::1564 > The time taken to retrun data serial number search::1585 > The time taken to retrun data serial number search::1586 > The time taken to retrun data serial number search::1712 > The time taken to retrun data serial number search::1713 > The time taken to retrun data serial number search::2047 > The time taken to retrun data serial number search::2048 > The time taken to retrun data serial number search::2073 > The time taken to retrun data serial number search::2074 > The time taken to retrun data serial number search::2065 > The time taken to retrun data serial number search::2066 > The time taken to retrun data serial number search::2875 > The time taken to retrun data serial number search::2876 > The time taken to retrun data serial number search::3065 > The time taken to retrun data serial number search::3065 > The time taken to retrun data serial number search::3068 > The time taken to retrun data serial number search::3069 > The time taken to retrun data serial number search::3070 > The time taken to retrun data serial number search::3075 > The time taken to retrun data serial number search::3071 > The time taken to retrun data serial number search::3076 > The time taken to retrun data serial number search::3092 > The time taken to retrun data serial number search::3093 > The time taken to retrun data serial number search::3529 > The time taken to retrun data serial number search::3530 > The time taken to retrun data serial number search::3570 > The time taken to retrun data serial number search::3570 > The time taken to retrun data serial number search::4124 > The time taken to retrun data serial number search::4125 > The time taken to retrun data serial number search::4152 > The time taken to retrun data serial number search::4153 > The time taken to retrun data serial number search::4153 > The time taken to retrun data serial number search::4154 > The time taken to retrun data serial number search::4149 > The time taken to retrun data serial number search::4150 > The time taken to retrun data serial number search::5165 > The time taken to retrun data serial number search::5165 > The time taken to retrun data serial number search::5134 > The time taken to retrun data serial number search::5135 > The time taken to retrun data serial number search::5146 > The time taken to retrun data serial number search::5146 > The time taken to retrun data serial number search::5291 > The time taken to retrun data serial number search::5292 > The time taken to retrun data serial number search::5290 > The time taken to retrun data serial number search::5291 > The time taken to retrun data serial number search::5639 > The time taken to retrun data serial number search::5640 > > Thanks, > Naresh.K -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/aa1d262e-959a-46f7-b6a7-4e9f7559983b%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/aa1d262e-959a-46f7-b6a7-4e9f7559983b%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:29am UTC](https://discuss.elastic.co/t/elasticsearch-performance-issue-with-load-increasing-reponse-time-is-increasing/16049/6 "2017-07-06T01:29:37Z")