we have implemented a ES cluster with 4 nodes; out of 4, 3 nodes are master eligible nodes and also those nodes are data nodes, 4th node is a data node and also marked as a ingest node.
We use this cluster for logs. We have one kibana instance for data visualization(currently 10 - 30 users).
up to how many users' traffic can be handled by this kibana instance?
And also, we are planing to scale this cluster for more nodes. what are the best practices that we should follow?
users count will increase in future(up to 100 - 200 users) also, in order to manage kibana traffic how many kibana instance we will need?
It depends on what the users are doing in Kibana. You mentioned that you're looking at logs and doing data visualization, which is not actually handled by the Kibana server- the bottleneck here will be Elasticsearch performance. There are some types of workloads that require more Kibana servers, for example:
Running many watcher/alerting tasks
Large reporting jobs
In general, you are probably okay with one Kibana instance. Add a second instance if it becomes a problem.
Hi @wylie ,
Thank you for your reply. Basically what we have setup is that we have 4 servers, and each has metribeat, packetbeat, auditbeat, filebeat instances and also mutiple filebeat instances and logstash. In between filebeat and logstash, we have kafka cluster. So, all the data poll to ES cluster. Recently, We have noticed that ES cluster load is become higher and higher and eventually, kibana response time become high. I was hoping to get an advice to avoid this. Do we need to scale the ES cluster?
Kibana is used for logs minitoring, and also for security(intrusion detection), etc. Any additional advice would be helpful.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
