I have two main questions. I read through a lo to documentation and forums, but still am unclear regarding how 1) queries are handled on a host that has Kibana talking locally to master ES node, and 2) whether I am better off changing that host to a coordinator node only.
I currently have a 6 VM node cluster 3 masters, 6 data nodes, 1 kibana.
each host has 8core CPUs and 32GBs of RAM with JVM Heap size set to 16GBs (unlimited high QoS disks)
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name x.x.x.x 56 81 8 4.00 3.99 3.88 md - node1 x.x.x.x 46 80 5 2.53 2.90 2.56 d - node2 x.x.x.x 36 77 3 2.69 2.73 2.79 md - node3 x.x.x.x 67 81 6 4.03 3.20 3.08 d - node4 x.x.x.x 61 83 6 3.14 3.30 3.48 md * node5 x.x.x.x 49 77 5 1.87 1.89 1.87 d - node6 shards disk.indices disk.used disk.avail disk.total disk.percent host ip node 140 1.8tb 2tb 2.1tb 4.1tb 48 x.x.x.x x.x.x.x node1 140 1.8tb 1.5tb 2.6tb 4.2tb 37 x.x.x.x x.x.x.x node2 141 1.9tb 1.6tb 3.6tb 5.3tb 31 x.x.x.x x.x.x.x node3 141 1.9tb 1.6tb 2.3tb 4tb 42 x.x.x.x x.x.x.x node4 140 1.5tb 1.9tb 4.4tb 6.4tb 30 x.x.x.x x.x.x.x node5 140 1.7tb 1.5tb 2.8tb 4.3tb 34 x.x.x.x x.x.x.x node6
It will be very hard for me to order additional hosts, so for now lets say I am stuck with just these 6 hosts.
kibana | | | ES Node1 ------ES Node2 ------ES Node3 ------ES Node4 -------ES Node5 -------ES Node6 ----- master: true master: true master: true master: false master: false master: false data: true data: true data: true data: true data: true data: true
currently when I load kibana the page takes about a minute to load. my users report querying is slow. queries are parsing huge number of application logs used in load-testing our environment. I never have worked with kibana especially for querying actual data so i dont know what should be the norm.
I am trying to improve the performance anywhere I can, and started thinking about load balancing queries. My feeling is that changing node1 to a coordinating host will increase querying as the coordinator behaves like a load balancer. Is the Kibana to master node not load balancing but executing all queries on that one master?
1st question: having the current setup of Kibana talking locally to a Master node, does this mean that all queries are handled by Node1, or are queries spread out amongst all nodes?
2nd question: is it generally expected that switching node1 to a kibana/coordinator only node will greatly increase querying?
kibana | | | ES Node1 ----------ES Node2 ------ES Node3 ----- ES Node4 -----ES Node5 ------ES Node6 ---- master: false master: true master: true master: true master: false master: false data: false data: true data: true data: true data: true data: true ingest: false (coordinator only)
I would appreciate if anyone could give me insight as to a comparison between Kibana talking to a master node vs kibana talking to a coordinator only node?
Secondly, any feedback/recommendations on the above would be greatly appreciated on any enhancements that could be made.
thanks in advance...