# Poor Query Performance Results **URL:** \ **Category:** Elasticsearch\ **Created:** [April 29, 2018, 12:41pm UTC](https://discuss.elastic.co/t/poor-query-performance-results/129954 "2018-04-29T12:41:09Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![groot](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/groot/32/20933_2.png) [@groot](https://discuss.elastic.co/u/groot)\ **Post date:** [April 29, 2018, 12:41pm UTC](https://discuss.elastic.co/t/poor-query-performance-results/129954/1 "2018-04-29T12:41:09Z")
Hi , I need advice , I have build in my company a new ELK environment with 5 nodes (vm machines) which one is Coordinator node with kibana And the other nodes are data nodes with 6 shards per index . Between our coordinator node and data node I have configured stunnel, When I'm sending a query to my ELK with curl command I'm get the 1st results from our elasticsearch in 47s , the 2nd results from the same query are better and its stable to 8-10s each. My hosts configuration is: 4 CPU with 43g Memory and JVM Xms&Xmx is 16g Any idea why? here is my coordinator configuration: cluster.name: elkcentral node.name: elkcoordinator path.data: /data path.logs: /logs transport.bind\_host: _local_ transport.publish\_port: 9900 http.bind\_host: _global_ http.port: 9200 node.master: false node.data: false node.ingest: false action.destructive\_requires\_name: true processors: 4 thread\_pool.search.size: 6 network.tcp.keep\_alive: true transport.ping\_schedule: 5s http.cors.enabled: true http.cors.allow-origin: "\*" discovery.zen.ping.unicast.hosts: ["localhost:9900", "localhost:9901", "localhost:9902", "localhost:9903","localhost:9904"] discovery.zen.minimum\_master\_nodes: 2 gateway.recover\_after\_nodes: 4 path.repo: ["/elk/snapshots"] http.type: ssl\_netty4 ---
**Author:** ![thiago](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thiago/32/32096_2.png) [@thiago](https://discuss.elastic.co/u/thiago)\ **Post date:** [April 30, 2018, 8:39am UTC](https://discuss.elastic.co/t/poor-query-performance-results/129954/2 "2018-04-30T08:39:09Z")
Are you running all the 6 nodes in vm machines within a single host with only 4 CPU and a total of 43GB RAM? Or is it that each vm has 4 CPU and 43GB? If its the case, what is total CPU cores and RAM of this host? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [April 30, 2018, 9:01am UTC](https://discuss.elastic.co/t/poor-query-performance-results/129954/3 "2018-04-30T09:01:58Z")
Could you provide the output of the [cluster stats API](https://www.elastic.co/guide/en/elasticsearch/reference/6.2/cluster-stats.html) so we can get a better understanding of what the cluster looks like? You mention using tunnel between the coordinating node and the rest of the nodes. Could you please elaborate on how the cluster is deployed and what load it is under? ---
**Author:** ![groot](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/groot/32/20933_2.png) [@groot](https://discuss.elastic.co/u/groot)\ **Post date:** [April 30, 2018, 2:15pm UTC](https://discuss.elastic.co/t/poor-query-performance-results/129954/4 "2018-04-30T14:15:10Z")
Hi @thiago, Thanks for the replay, Regarding your question I have 5 hosts and not 6 each host has 41g memory and 4 cpu. Total Cores 5 host \* 4 Core = 20 cpu total Total Memory 5 Host \* 41G = 205g memory total ---
**Author:** ![thiago](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thiago/32/32096_2.png) [@thiago](https://discuss.elastic.co/u/thiago)\ **Post date:** [April 30, 2018, 3:59pm UTC](https://discuss.elastic.co/t/poor-query-performance-results/129954/5 "2018-04-30T15:59:03Z")
That should not be a problem then. Can you please attach what Christian has requested? ---
**Author:** ![groot](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/groot/32/20933_2.png) [@groot](https://discuss.elastic.co/u/groot)\ **Post date:** [May 1, 2018, 11:18am UTC](https://discuss.elastic.co/t/poor-query-performance-results/129954/6 "2018-05-01T11:18:14Z")
Hi @Christian_Dahlqvist , The connection is secured with stunnel here is the stunnel conf example 🙂 [es-http-local-server] client = no accept = 19200 connect = localhost:9200 CAfile = /etc/ssl/certs/elk\_certificate.pem verify = 2 [es-transport-co-ord-client] client = yes accept = localhost:9900 connect = elk:19300 [es-transport-node01-client] client = yes accept = localhost:9901 connect = elkdp01:19300 [es-transport-node02-client] client = yes accept = localhost:9902 connect = elkdp02:19300 .... The coordinator node is listen to network connection for rsyslog, logstash and kibana connection While most of the servers sending data via rsyslog service to our rsyslog server and then forward it to my coordinator, when query any thing it's direct to my coordinator node, here is the output from my elastic cluster { "\_nodes" : { "total" : 5, "successful" : 5, "failed" : 0 }, "cluster\_name" : "elk\_centralized\_logger", "timestamp" : 1525097008481, "status" : "green", "indices" : { "count" : 136, "shards" : { "total" : 1280, "primaries" : 640, "replication" : 1.0, "index" : { "shards" : { "min" : 2, "max" : 12, "avg" : 9.411764705882353 }, "primaries" : { "min" : 1, "max" : 6, "avg" : 4.705882352941177 }, "replication" : { "min" : 1.0, "max" : 1.0, "avg" : 1.0 } } }, "docs" : { "count" : 1143442501, "deleted" : 59054 }, "store" : { "size" : "720.3gb", "size\_in\_bytes" : 773488810938, "throttle\_time" : "0s", "throttle\_time\_in\_millis" : 0 }, "fielddata" : { "memory\_size" : "0b", "memory\_size\_in\_bytes" : 0, "evictions" : 0 }, "query\_cache" : { "memory\_size" : "938.4mb", "memory\_size\_in\_bytes" : 984046371, "total\_count" : 421545, "hit\_count" : 95625, "miss\_count" : 325920, "cache\_size" : 35898, "cache\_count" : 36867, "evictions" : 969 }, "completion" : { "size" : "0b", "size\_in\_bytes" : 0 }, "segments" : { "count" : 13381, "memory" : "2.3gb", "memory\_in\_bytes" : 2535474517, "terms\_memory" : "1.9gb", "terms\_memory\_in\_bytes" : 2089938623, "stored\_fields\_memory" : "321.9mb", "stored\_fields\_memory\_in\_bytes" : 337610384, "term\_vectors\_memory" : "0b", "term\_vectors\_memory\_in\_bytes" : 0, "norms\_memory" : "4.6mb", "norms\_memory\_in\_bytes" : 4874752, "points\_memory" : "42.1mb", "points\_memory\_in\_bytes" : 44205474, "doc\_values\_memory" : "56.1mb", "doc\_values\_memory\_in\_bytes" : 58845284, "index\_writer\_memory" : "5.7mb", "index\_writer\_memory\_in\_bytes" : 6008504, "version\_map\_memory" : "903b", "version\_map\_memory\_in\_bytes" : 903, "fixed\_bit\_set" : "271.6mb", "fixed\_bit\_set\_memory\_in\_bytes" : 284844488, "max\_unsafe\_auto\_id\_timestamp" : 9223372036854775807, "file\_sizes" : { } } }, "nodes" : { "count" : { "total" : 5, "data" : 4, "coordinating\_only" : 1, "master" : 3, "ingest" : 4 }, "versions" : [ "5.6.5" ], "os" : { "available\_processors" : 20, "allocated\_processors" : 20, "names" : [ { "name" : "Linux", "count" : 5 } ], "mem" : { "total" : "205.5gb", "total\_in\_bytes" : 220687183872, "free" : "4gb", "free\_in\_bytes" : 4298481664, "used" : "201.5gb", "used\_in\_bytes" : 216388702208, "free\_percent" : 2, "used\_percent" : 98 } }, "process" : { "cpu" : { "percent" : 17 }, "open\_file\_descriptors" : { "min" : 324, "max" : 1104, "avg" : 843 } }, "jvm" : { "max\_uptime" : "39d", "max\_uptime\_in\_millis" : 3374293868, "versions" : [ { "version" : "1.8.0\_161", "vm\_name" : "Java HotSpot(TM) 64-Bit Server VM", "vm\_version" : "25.161-b12", "vm\_vendor" : "Oracle Corporation", "count" : 5 } ], "mem" : { "heap\_used" : "43.3gb", "heap\_used\_in\_bytes" : 46522328000, "heap\_max" : "79.8gb", "heap\_max\_in\_bytes" : 85725020160 }, "threads" : 342 }, "fs" : { "total" : "787.3gb", "total\_in\_bytes" : 845377593344, "free" : "590.5gb", "free\_in\_bytes" : 634127507456, "available" : "550.5gb", "available\_in\_bytes" : 591161057280, "spins" : "true" }, "plugins" : [], "network\_types" : { "transport\_types" : { "netty4" : 5 }, "http\_types" : { "ssl\_netty4" : 1, "netty4" : 4 } } } } hope that I did not forgot anything, ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [May 1, 2018, 11:40am UTC](https://discuss.elastic.co/t/poor-query-performance-results/129954/7 "2018-05-01T11:40:20Z")
What type of queries are you running that are taking that long? How much are you indexing into the cluster each day? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 29, 2018, 11:40am UTC](https://discuss.elastic.co/t/poor-query-performance-results/129954/8 "2018-05-29T11:40:33Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.