Kibana --> ES master vs ES Coordinating Nodes in Cluster Question

Hi all,

I have two main questions. I read through a lo to documentation and forums, but still am unclear regarding how 1) queries are handled on a host that has Kibana talking locally to master ES node, and 2) whether I am better off changing that host to a coordinator node only.

I currently have a 6 VM node cluster 3 masters, 6 data nodes, 1 kibana.
each host has 8core CPUs and 32GBs of RAM with JVM Heap size set to 16GBs (unlimited high QoS disks)

ip          heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
x.x.x.x               56          81   8    4.00    3.99     3.88 md        -      node1
x.x.x.x               46          80   5    2.53    2.90     2.56 d         -      node2
x.x.x.x               36          77   3    2.69    2.73     2.79 md        -      node3
x.x.x.x               67          81   6    4.03    3.20     3.08 d         -      node4
x.x.x.x               61          83   6    3.14    3.30     3.48 md        *      node5
x.x.x.x               49          77   5    1.87    1.89     1.87 d         -      node6

shards disk.indices disk.used disk.avail disk.percent host        ip          node
   140        1.8tb       2tb      2.1tb      4.1tb           48 x.x.x.x     x.x.x.x     node1
   140        1.8tb     1.5tb      2.6tb      4.2tb           37 x.x.x.x     x.x.x.x     node2
   141        1.9tb     1.6tb      3.6tb      5.3tb           31 x.x.x.x     x.x.x.x     node3
   141        1.9tb     1.6tb      2.3tb        4tb           42 x.x.x.x     x.x.x.x     node4
   140        1.5tb     1.9tb      4.4tb      6.4tb           30 x.x.x.x     x.x.x.x     node5
   140        1.7tb     1.5tb      2.8tb      4.3tb           34 x.x.x.x     x.x.x.x     node6

It will be very hard for me to order additional hosts, so for now lets say I am stuck with just these 6 hosts.

Current setup:

ES Node1 ------ES Node2 ------ES Node3 ------ES Node4 -------ES Node5 -------ES Node6 -----
master: true   master: true   master: true   master: false   master: false   master: false         
data: true     data: true     data: true     data: true      data: true      data: true

currently when I load kibana the page takes about a minute to load. my users report querying is slow. queries are parsing huge number of application logs used in load-testing our environment. I never have worked with kibana especially for querying actual data so i dont know what should be the norm.

I am trying to improve the performance anywhere I can, and started thinking about load balancing queries. My feeling is that changing node1 to a coordinating host will increase querying as the coordinator behaves like a load balancer. Is the Kibana to master node not load balancing but executing all queries on that one master?

1st question: having the current setup of Kibana talking locally to a Master node, does this mean that all queries are handled by Node1, or are queries spread out amongst all nodes?

2nd question: is it generally expected that switching node1 to a kibana/coordinator only node will greatly increase querying?


ES Node1 ----------ES Node2 ------ES Node3 ----- ES Node4 -----ES Node5 ------ES Node6 ----
master: false      master: true   master: true   master: true  master: false  master: false         
data: false        data: true     data: true     data: true    data: true     data: true           
ingest: false
(coordinator only)

I would appreciate if anyone could give me insight as to a comparison between Kibana talking to a master node vs kibana talking to a coordinator only node?

Secondly, any feedback/recommendations on the above would be greatly appreciated on any enhancements that could be made.

thanks in advance...

If you have coordinating nodes, then all queries should go via them.

By default all nodes are coordinating ones, in that if there is data it needs to query that it doesn't have, it will ask the node that has the data for the answer.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.