Elastic profiler timings - Time taken to merge search results?

[Mahesh_Hegde]

How can we know the time taken to merge search results returned by all shards?

I have enabled the search.slowlog.threshold.query.debug settings , but this gives shard level timings only. Even profile : true in query, returns shard level timings.

My questions:
[1] Can we relate query, rewrite and collector times to merge time ? how are they related?
[2] How exactly can we calculate merge time with best approximation or nearest/exact value?

-Mahesh

[polyfractal (Zachary Tong)]

First, make sure you've read the Profiler API docs and this recent article since they discuss what some of the timings mean.

Now, to your question, merge time isn't currently profiled. It's essentially just inserting values into a priority queue of size n, using the shard-local score as the sort value for the priority queue. I don't quite remember how ES implements it, but priority queues are usually implemented as heaps. So insertion/removal is probably O(log n)

I would not expect the merge time of hits to be a significant contributor to latency, especially relative to actually scoring documents and touching the disk. But it'd be nice if we could add it to the profiler eventually.

Now, if you're talking about merging aggregations together (e.g. in map-reduce fashion), that is timed under the reduce method. See: https://www.elastic.co/guide/en/elasticsearch/reference/current/_profiling_aggregations.html for more details