How does search latency and index latency calculate on marvel's dashborad

[makeyang]

currently the latency is calcaluted by marvel is not fit with the valued we calculated from client side. so we'd like to konw how it calculated by marvel

[pickypg (Chris Earle)]

Hi,

I recently explained the calculation for the latency over at:

Negative Search Latency in Marvel

Hi Zaar Hai, I'd like to tell you that we're predictively pre-searching for you, but this is a visualization bug that comes from how we calculate the the latency: Search Latency is calculated by dividing the derivative of the total search time by the derivative of the total search count. Index Latency is the same thing. The derivative is necessary because the stats reported from Elasticsearch itself are just counters since the node started, so each time bucket (x coordinate) gets this extra st…

The gist is that we calculate latency by pulling the _nodes/stats API, and fetching the total time spent indexing and the number of indexing operations, as well as the same for query (search). Those operations are per shard to be clear. We do this every 10s, so it slowly grows over time by adding to the initial value since node uptime.

So, to chart it, we take the derivative for each time-bucket (each X-axis slice) that we intend to chart, then divide those derivatives to get the latency.

Keep in mind that your client side latency is going to have more data (the full network round trip for starters), so it is inherently a little more accurate.

Hope that helps,
Chris