Hi,
The same metrics search.query_time_in_millis, search.query_total are provided by Node API and Index API, how they are calculated? is it just a sum of shards metrics search.query_time_in_millis, search.query_total, located on the same node (for Node API) and sum of shards of the same index (Index API)? Should there be some relation between metrics search.query_time_in_millis, search.query_total metrics provided by Node API and search.query_time_in_millis provided by Index API?
To visualize query time, I use this dashboard https://grafana.com/grafana/dashboards/2347, query for "Query Time":
rate(elasticsearch_indices_search_query_time_seconds{cluster=\"$cluster\",node=~\"$node\", node_type=~\"$node_type\"}[$interval])
(this metric is coming from elasticsearch-exporter https://github.com/justwatchcom/elasticsearch_exporter/blob/master/collector/nodes.go#L602-L612)
does reported by Node API metric search.query_time_in_millis include somehow search.query_total , or query should be changed to
rate(elasticsearch_indices_search_query_time_seconds{cluster=\"$cluster\",node=~\"$node\", node_type=~\"$node_type\"}[$interval]) / rate(elasticsearch_indices_search_query_total{cluster=\"$cluster\",node=~\"$node\", node_type=~\"$node_type\"}[$interval])
?