Monitoring/debugging queries per index

I'd like to try to get some insight into the kinds of queries folks are running through kibana, as well as figure out what kind of impacts those queries might be having on the underlying elasticsearch clusters.

A simple example would be someone making a dashboard chart with well over 100 separate lines. Currently, that dashboard query times out on any time range larger than 1 hour. I'd love to be able to quickly look and say "This specific query is timing out at the elasticsearch query level (Searching some shards was too slow)" vs "This is timing out/overwhelming the kibana node" vs "This specific query is timing out at the aggregation step of the query (Gotten all results from all shards, but it took too long/too much memory to aggregate and order them)". Right now, I have to do a lot of thinking and massaging of elasticsearch logs/metrics to try to infer the impact of queries, which works but doesn't scale very well.

If not a view of specific queries, is there at least some way to view overall query stats on individual shards or indices? e.g. how many queries hit index A vs index B per day, or better yet, how long/how much CPU/how much memory are queries for index A taking per day.

Are you aware of the slow log?

I was not! Slowlog seems to cover the "What's going on with these slow queries" case pretty perfectly. Thanks a ton!

For closure on the latter usecase of getting query stats for individual indices, I feel silly for even asking, as it's probably accomplishable with the stats api. Somehow I missed that.

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.