Hi all,
I have two main questions. I read through a lo to documentation and forums, but still am unclear regarding how 1) queries are handled on a host that has Kibana talking locally to master ES node, and 2) whether I am better off changing that host to a coordinator node only.
I currently have a 6 VM node cluster 3 masters, 6 data nodes, 1 kibana.
each host has 8core CPUs and 32GBs of RAM with JVM Heap size set to 16GBs (unlimited high QoS disks)
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
x.x.x.x 56 81 8 4.00 3.99 3.88 md - node1
x.x.x.x 46 80 5 2.53 2.90 2.56 d - node2
x.x.x.x 36 77 3 2.69 2.73 2.79 md - node3
x.x.x.x 67 81 6 4.03 3.20 3.08 d - node4
x.x.x.x 61 83 6 3.14 3.30 3.48 md * node5
x.x.x.x 49 77 5 1.87 1.89 1.87 d - node6
shards disk.indices disk.used disk.avail disk.total disk.percent host ip node
140 1.8tb 2tb 2.1tb 4.1tb 48 x.x.x.x x.x.x.x node1
140 1.8tb 1.5tb 2.6tb 4.2tb 37 x.x.x.x x.x.x.x node2
141 1.9tb 1.6tb 3.6tb 5.3tb 31 x.x.x.x x.x.x.x node3
141 1.9tb 1.6tb 2.3tb 4tb 42 x.x.x.x x.x.x.x node4
140 1.5tb 1.9tb 4.4tb 6.4tb 30 x.x.x.x x.x.x.x node5
140 1.7tb 1.5tb 2.8tb 4.3tb 34 x.x.x.x x.x.x.x node6
It will be very hard for me to order additional hosts, so for now lets say I am stuck with just these 6 hosts.
Current setup:
kibana
|
|
|
ES Node1 ------ES Node2 ------ES Node3 ------ES Node4 -------ES Node5 -------ES Node6 -----
master: true master: true master: true master: false master: false master: false
data: true data: true data: true data: true data: true data: true
currently when I load kibana the page takes about a minute to load. my users report querying is slow. queries are parsing huge number of application logs used in load-testing our environment. I never have worked with kibana especially for querying actual data so i dont know what should be the norm.
I am trying to improve the performance anywhere I can, and started thinking about load balancing queries. My feeling is that changing node1 to a coordinating host will increase querying as the coordinator behaves like a load balancer. Is the Kibana to master node not load balancing but executing all queries on that one master?
1st question: having the current setup of Kibana talking locally to a Master node, does this mean that all queries are handled by Node1, or are queries spread out amongst all nodes?
2nd question: is it generally expected that switching node1 to a kibana/coordinator only node will greatly increase querying?
Proposed:
kibana
|
|
|
ES Node1 ----------ES Node2 ------ES Node3 ----- ES Node4 -----ES Node5 ------ES Node6 ----
master: false master: true master: true master: true master: false master: false
data: false data: true data: true data: true data: true data: true
ingest: false
(coordinator only)
I would appreciate if anyone could give me insight as to a comparison between Kibana talking to a master node vs kibana talking to a coordinator only node?
Secondly, any feedback/recommendations on the above would be greatly appreciated on any enhancements that could be made.
thanks in advance...