Query statistics difference

[sunilmchaudhari (Sunil Chaudhari)]

Hi,

I have queried data from Kibana and took 3 samples of statistics for the same query.

I have got 3 different results.

Query Duration 2366ms
Request Duration 2502ms
Hits 3
Index -

Query Duration 2346ms
Request Duration 2901ms
Hits 3
Index -

Query Duration 2588ms
Request Duration 2719ms
Hits 3
Index -

Can somebody explain why the difference in this?
What parameter matters for improving the request and response time?

br,
Sunil

[Christian_Dahlqvist]

There are quite a few parameters that can affect the performance of a Kibana dashboard. Off the top of my head I would list the following:

1. Number and types of visualisations. Each visualisations will typically result in one aggregation, so more visualisations mean more work for Elasticsearch. Different types of aggregations may have different performance profiles.
2. Amount of data aggregated over. This is typically determined by the number of indices matching the index pattern and holding data for the time period aggregated across.
3. Size and performance of the Elasticsearch cluster. Large aggregations can often be limited by the amount of available CPU and/or IO resources available. Number of available CPU cores and the speed and performance of the storage used will have an impact on this. The number of data nodes in the cluster determines the total amount of resources available.
4. Size and number of shards being queried. The minimum query duration is often determined by the shard size as each aggregation is single threaded across each shard. It is important to identify a good balance between shard size and number of shards. Having too large or too small shards can hurt performance.
5. Overall load on the cluster. Indexing and querying use the same resources and will generally affect each other.

When sizing a cluster, I usually recommend benchmarking it with a realistic mix of indexing and querying to ensure that latency requirements for querying are met. There is a talk about sizing methodology available here.