### Author:  [@hackerwin7](https://discuss.elastic.co/u/hackerwin7)
#### Post date: [August 23, 2019, 10:12am UTC](https://discuss.elastic.co/t/7-2-0-spend-more-time-on-query-next-doc-and-aggregation-collect-than-6-8-0/196489/1 "2019-08-23T10:12:19Z")
Recently，we upgrade our es cluster to 7.2.0 and find the same search took time is longer than 6.8.0
.
so we prepare for two cluster (7.2 and 6.8) with same index data set , same mapping, and execute same search. we found that 7.2.0 is slower than 6.8.0,
our tests below:
7.2.0 index list:
6.8.0 index list :
doc count is same across two clusters
profile the same search:
>
After reviewing these profile results, I find that 7.2.0 took more time on **advance** in query and **collect** in aggregation
Is the Lucene upgrade cause this performance drop?
review the aggregation profile section, some phase of 7.x is extremely slow:
```auto
a shard aggregation profile instance
| version | collect | collect_count | build_aggregation | build_aggregation_count |
|---------|----------|---------------|-------------------|-------------------------|
| 7.2 | 63333540 | 632532 | 89351401 | 4 |
| 6.8 | 40441073 | 632532 | 52753450 | 4 |
```
---
### Author:  [@Bertrand](https://discuss.elastic.co/u/Bertrand)
#### Post date: [August 23, 2019, 10:44am UTC](https://discuss.elastic.co/t/7-2-0-spend-more-time-on-query-next-doc-and-aggregation-collect-than-6-8-0/196489/2 "2019-08-23T10:44:32Z")
Could this be similar to [Slow date\_histogram after upgrading to 7.3.0 on "dense" indexes](https://discuss.elastic.co/t/slow-date-histogram-after-upgrading-to-7-3-0-on-dense-indexes/196475/2) ?
---
### Author:  [@hackerwin7](https://discuss.elastic.co/u/hackerwin7)
#### Post date: [August 23, 2019, 12:22pm UTC](https://discuss.elastic.co/t/7-2-0-spend-more-time-on-query-next-doc-and-aggregation-collect-than-6-8-0/196489/3 "2019-08-23T12:22:16Z")
@Bertrand
Maybe, I'm not sure the root reason is? Did you compare your advance and advance\_count between two versions?
---
### Author:  [@Bertrand](https://discuss.elastic.co/u/Bertrand)
#### Post date: [August 23, 2019, 2:28pm UTC](https://discuss.elastic.co/t/7-2-0-spend-more-time-on-query-next-doc-and-aggregation-collect-than-6-8-0/196489/4 "2019-08-23T14:28:22Z")
What do you mean by `advance` and `advance_count`?
---
### Author:  [@hackerwin7](https://discuss.elastic.co/u/hackerwin7)
#### Post date: [August 25, 2019, 2:58am UTC](https://discuss.elastic.co/t/7-2-0-spend-more-time-on-query-next-doc-and-aggregation-collect-than-6-8-0/196489/5 "2019-08-25T02:58:36Z")
@Bertrand
see [this](https://www.elastic.co/guide/en/elasticsearch/reference/7.2/search-profile-queries.html#_all_parameters_3)
---
### Author:  [@hackerwin7](https://discuss.elastic.co/u/hackerwin7)
#### Post date: [August 26, 2019, 3:15am UTC](https://discuss.elastic.co/t/7-2-0-spend-more-time-on-query-next-doc-and-aggregation-collect-than-6-8-0/196489/6 "2019-08-26T03:15:54Z")
Any benchmark reports for these versions?
---
### Author:  [@Bertrand](https://discuss.elastic.co/u/Bertrand)
#### Post date: [August 26, 2019, 12:24pm UTC](https://discuss.elastic.co/t/7-2-0-spend-more-time-on-query-next-doc-and-aggregation-collect-than-6-8-0/196489/7 "2019-08-26T12:24:45Z")
I made an extra run with the "uniform-sameday" dataset and query profiling enabled.
To recap:
- `70m` docs, each with a single `@timestamp` field - dates are all in the same day, evenly distributed on 24h
- 3 nodes, 3 shards, no replica
Results for both `6.8.2` and `7.3.0` are as follows:
```auto
timing advance advance_count
shard1, shard2, shard3 shard1, shard2, shard3
--------------------------------------------------------------------
6.8.2 6705 0, 0, 0 0, 0, 0
7.3.0 10130 26094, 14987, 14778 23, 18, 16
```
I have no idea what `advance` and `advance_count` represent - but values are not the same for both versions.
Complete query profile results are available at:
- `6.8.2`: [https://gist.github.com/brenuart/a048df2132f13ebf70ffac572f30b5a0#file-6-8-2](https://gist.github.com/brenuart/a048df2132f13ebf70ffac572f30b5a0#file-6-8-2)
- `7.3.0`: [https://gist.github.com/brenuart/a048df2132f13ebf70ffac572f30b5a0#file-7-3-0](https://gist.github.com/brenuart/a048df2132f13ebf70ffac572f30b5a0#file-7-3-0)
The query was:
```auto
{
"profile": true,
"aggs": {
"2": {
"date_histogram": {
"field": "@timestamp",
"interval": "1d",
"min_doc_count": 1
}
}
},
"query": {
"match_all": {}
}
}
```
---
### Author:  [@hackerwin7](https://discuss.elastic.co/u/hackerwin7)
#### Post date: [August 27, 2019, 6:12am UTC](https://discuss.elastic.co/t/7-2-0-spend-more-time-on-query-next-doc-and-aggregation-collect-than-6-8-0/196489/8 "2019-08-27T06:12:00Z")
@Bertrand
Similar to my case, but I want to know why 7.x will have more `advance` cost than 6.8, is it for more accurate scorer for Lucene docs?
---
### Author:  [@hackerwin7](https://discuss.elastic.co/u/hackerwin7)
#### Post date: [August 28, 2019, 8:22am UTC](https://discuss.elastic.co/t/7-2-0-spend-more-time-on-query-next-doc-and-aggregation-collect-than-6-8-0/196489/9 "2019-08-28T08:22:01Z")
@Bertrand
I remove the date\_histogram in my search, just use terms aggregation for both two clusters; the 7.2 search took time is still longer than 6.8 search took
---
### Author:  [@ted\_ye](https://discuss.elastic.co/u/ted_ye)
#### Post date: [September 10, 2019, 9:05am UTC](https://discuss.elastic.co/t/7-2-0-spend-more-time-on-query-next-doc-and-aggregation-collect-than-6-8-0/196489/10 "2019-09-10T09:05:27Z")
@hackerwin7
hello, have you got the answer of this problem?
I've met the same problem, the same machine, the same index, different version of elasticsearch, 7.3.1 vs 6.2.4 ,when I search the logs on kibana with the same query request, It takes longer on 7.3.1 than 6.2.4.
---
### Author:  [@hackerwin7](https://discuss.elastic.co/u/hackerwin7)
#### Post date: [September 10, 2019, 10:26am UTC](https://discuss.elastic.co/t/7-2-0-spend-more-time-on-query-next-doc-and-aggregation-collect-than-6-8-0/196489/11 "2019-09-10T10:26:59Z")
@ted_ye
I still have no idea for this, I think the problem is in Aggregator.getCollector().collect() function, the 7.x spend more time in collect() in Lucene query phase
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [September 11, 2019, 6:48am UTC](https://discuss.elastic.co/t/7-2-0-spend-more-time-on-query-next-doc-and-aggregation-collect-than-6-8-0/196489/12 "2019-09-11T06:48:04Z")
@jimczi any idea?
---
### Author:  [@jimczi](https://discuss.elastic.co/u/jimczi)
#### Post date: [September 12, 2019, 12:27pm UTC](https://discuss.elastic.co/t/7-2-0-spend-more-time-on-query-next-doc-and-aggregation-collect-than-6-8-0/196489/13 "2019-09-12T12:27:15Z")
This is related to the migration of Joda to Java time. We're investigating the slowdown (that we are now able to reproduce easily) in [https://github.com/elastic/elasticsearch/issues/45702#issuecomment-530756419](https://github.com/elastic/elasticsearch/issues/45702#issuecomment-530756419) and we have an idea on how to restore the performance of 6.8. Stay tuned 😉
---
### Author:  [@hackerwin7](https://discuss.elastic.co/u/hackerwin7)
#### Post date: [September 12, 2019, 1:46pm UTC](https://discuss.elastic.co/t/7-2-0-spend-more-time-on-query-next-doc-and-aggregation-collect-than-6-8-0/196489/14 "2019-09-12T13:46:55Z")
@jimczi
nice catch for this! interested to follow the issue
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 10, 2019, 1:53pm UTC](https://discuss.elastic.co/t/7-2-0-spend-more-time-on-query-next-doc-and-aggregation-collect-than-6-8-0/196489/15 "2019-10-10T13:53:42Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.