Help to improve Kibana visuals performance

[elk_user1]

Hi,

We have an ES cluster with 4 nodes - 3 data nodes of which 2 act as master node also and 1 coordinating node on which Kibana is configured.

We are loading data into Elasticsearch index from Kafka using confluent elasticsearch sink connector. Considering daily indices, the total data we currently have is 44 indices and 110 shards(90GB size, 177 million documents)

While checking on performance of Kibana visualizations, it is taking around 20seconds to load. It would be great if you can help in improving this performance as expected response time for Kibana visuals is less than 5 seconds.

Thanks in advance.

[spalger (Spencer Alger)]

@elk_user1

While checking on performance of Kibana visualizations, it is taking around 20seconds to load.

I suggest trying out elasticsearch's explain API to see why you queries are running slowly. You might need to choose a more optimized mapping for some of the fields you are using. If you look in the spy panel on your visualizations you will be able to see the request they are sending to elasticsearch so you can try them yourself with the dev tools/console app.

[elk_user1]

Thanks for replying. Yes I have checked the query which is generated from Kibana and tried it in Elasticsearch, and it takes around 20 seconds. Let me see what explain API returns for those query. Our index documents are basically json and we are manually creating mapping using index template before loading data.

[elk_user1]

With cache enabled(default) and not much of the data flowing into the index, the visualization is loading in around 1300 ms. However if I add the same visualization to a dashboard (just single visualization in dashboard), it takes around 8 secs. Why is such a performance difference for same visualization ?