# Search performance difference between from&size and search&after **URL:** \ **Category:** Elasticsearch\ **Created:** [September 6, 2021, 5:03am UTC](https://discuss.elastic.co/t/search-performance-difference-between-from-size-and-search-after/283400 "2021-09-06T05:03:47Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![MarvinLiu](https://avatars.discourse-cdn.com/v4/letter/m/ccd318/32.png) [@MarvinLiu](https://discuss.elastic.co/u/MarvinLiu)\ **Post date:** [September 6, 2021, 5:03am UTC](https://discuss.elastic.co/t/search-performance-difference-between-from-size-and-search-after/283400/1 "2021-09-06T05:03:47Z")
There are hundreds of millions of documents in my index. When I search, I find that search&after is much slower than from&size。 Use from&size，search is quick，took several ms return，but use search after，it took 20 seconds。Search is sort by **time** and **key** (a copy of \_id) ，Why？what's the difference? search cmd: ```auto { "query":{ "bool":{ "filter":[ { "query_string":{ "query":"*" } }, { "range":{ " __time__":{ "gte":1324958207, "lte":1724958207 } } } ] } }, "size":10, "sort":[ { " __time__":{ "order":"desc" } }, { " __key__":{ "order":"desc" } } ], "search_after":[ 1630594662000, "6130e666-2-67e9e3-f5-1" ], "profile":true } ``` profile: ```auto "searches":[ { "query":[ { "type":"BoostQuery", "description":"(ConstantScore(DocValuesFieldExistsQuery [field= __time__]))^0.0", "time_in_nanos":45722536283, "breakdown":{ "set_min_competitive_score_count":0, "match_count":0, "shallow_advance_count":0, "set_min_competitive_score":0, "next_doc":45722241414, "match":0, "next_doc_count":410919487, "score_count":0, "compute_max_score_count":0, "compute_max_score":0, "advance":19517, "advance_count":39, "score":0, "build_scorer_count":78, "create_weight":14271, "shallow_advance":0, "create_weight_count":1, "build_scorer":261081 }, "children":[ { "type":"DocValuesFieldExistsQuery", "description":"DocValuesFieldExistsQuery [field= __time__]", "time_in_nanos":16571715415, "breakdown":{ "set_min_competitive_score_count":0, "match_count":0, "shallow_advance_count":0, "set_min_competitive_score":0, "next_doc":16571493898, "match":0, "next_doc_count":410919487, "score_count":0, "compute_max_score_count":0, "compute_max_score":0, "advance":15074, "advance_count":39, "score":0, "build_scorer_count":78, "create_weight":517, "shallow_advance":0, "create_weight_count":1, "build_scorer":205926 } } ] } ], "rewrite_time":116538, "collector":[ { "name":"PagingFieldCollector", "reason":"search_top_hits", "time_in_nanos":30851166561 } ] } ], "aggregations":[ ] } ``` ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [September 6, 2021, 8:06am UTC](https://discuss.elastic.co/t/search-performance-difference-between-from-size-and-search-after/283400/2 "2021-09-06T08:06:22Z")
just to be sure, have you compared both searches after a warmup period **and** both searches when compared either include or both do not include the `profile` boolean set? Also, have you compared the profile output of both (there is a tool in kibana dev-tools that makes the output a little more graphical and easier to read). In general using search after should require much less memory when paginated deeper (on the coordinating node), but both operations have to execute the same search. I also guess the query\_string in there is just an example, as otherwise you could go with a `match_all` query or none as well? Also, which Elasticsearch version are you on? ---
**Author:** ![MarvinLiu](https://avatars.discourse-cdn.com/v4/letter/m/ccd318/32.png) [@MarvinLiu](https://discuss.elastic.co/u/MarvinLiu)\ **Post date:** [September 6, 2021, 10:59am UTC](https://discuss.elastic.co/t/search-performance-difference-between-from-size-and-search-after/283400/3 "2021-09-06T10:59:30Z")
Thanks for reply，Elasticsearch version is 7.10. Query string is just an example, match\_all same result . ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [September 6, 2021, 2:32pm UTC](https://discuss.elastic.co/t/search-performance-difference-between-from-size-and-search-after/283400/4 "2021-09-06T14:32:08Z")
can you please answer the other questions above as well to get a better picture? Thank you! ---
**Author:** ![MarvinLiu](https://avatars.discourse-cdn.com/v4/letter/m/ccd318/32.png) [@MarvinLiu](https://discuss.elastic.co/u/MarvinLiu)\ **Post date:** [September 7, 2021, 5:11am UTC](https://discuss.elastic.co/t/search-performance-difference-between-from-size-and-search-after/283400/5 "2021-09-07T05:11:01Z")
![image](https://us1.discourse-cdn.com/elastic/original/3X/9/7/97b217d4389da6aa8b14e3d0af40d99f65b2f2ef.png) left result is without search-after，right result is with search after。 query type is different， BooleanQuery vs boostQuery。 description is different，left is DocValuesFieldExistsQuery&LongDistanceFeatureQuery，right is just DocValuesFieldExistsQuery。 ![image](https://us1.discourse-cdn.com/elastic/original/3X/d/7/d7118d9bb8f67cc78a88af49bc134308a0e1f878.png) children query also has some difference。 ---
**Author:** ![MarvinLiu](https://avatars.discourse-cdn.com/v4/letter/m/ccd318/32.png) [@MarvinLiu](https://discuss.elastic.co/u/MarvinLiu)\ **Post date:** [September 7, 2021, 5:14am UTC](https://discuss.elastic.co/t/search-performance-difference-between-from-size-and-search-after/283400/6 "2021-09-07T05:14:35Z")
full left profile, with out search-after ```auto -------------------- "searches":[ { "query":[ { "type":"BooleanQuery", "description":"#(ConstantScore(DocValuesFieldExistsQuery [field= __time__]))^0.0 LongDistanceFeatureQuery(field=,origin=1630594662000,pivotDistance=43183000)", "time_in_nanos":50216920945, "breakdown":{ "set_min_competitive_score_count":5076, "match_count":0, "shallow_advance_count":0, "set_min_competitive_score":615806781, "next_doc":20485284262, "match":0, "next_doc_count":193283578, "score_count":193283578, "compute_max_score_count":0, "compute_max_score":0, "advance":60903, "advance_count":39, "score":29114449991, "build_scorer_count":78, "create_weight":52360, "shallow_advance":0, "create_weight_count":1, "build_scorer":1266648 }, "children":[ { "type":"BoostQuery", "description":"(ConstantScore(DocValuesFieldExistsQuery [field= __time__]))^0.0", "time_in_nanos":19146981415, "breakdown":{ "set_min_competitive_score_count":0, "match_count":0, "shallow_advance_count":0, "set_min_competitive_score":0, "next_doc":0, "match":0, "next_doc_count":0, "score_count":0, "compute_max_score_count":0, "compute_max_score":0, "advance":19146741485, "advance_count":193511946, "score":0, "build_scorer_count":117, "create_weight":25754, "shallow_advance":0, "create_weight_count":1, "build_scorer":214176 }, "children":[ { "type":"DocValuesFieldExistsQuery", "description":"DocValuesFieldExistsQuery [field= __time__]", "time_in_nanos":6894905065, "breakdown":{ "set_min_competitive_score_count":0, "match_count":0, "shallow_advance_count":0, "set_min_competitive_score":0, "next_doc":0, "match":0, "next_doc_count":0, "score_count":0, "compute_max_score_count":0, "compute_max_score":0, "advance":6894734027, "advance_count":193511946, "score":0, "build_scorer_count":117, "create_weight":625, "shallow_advance":0, "create_weight_count":1, "build_scorer":170413 } } ] }, { "type":"LongDistanceFeatureQuery", "description":"LongDistanceFeatureQuery(field=,origin=1630594662000,pivotDistance=43183000)", "time_in_nanos":19735814288, "breakdown":{ "set_min_competitive_score_count":5076, "match_count":0, "shallow_advance_count":117, "set_min_competitive_score":614826804, "next_doc":0, "match":0, "next_doc_count":0, "score_count":193283578, "compute_max_score_count":78, "compute_max_score":4210, "advance":7497803263, "advance_count":193283611, "score":11622998714, "build_scorer_count":78, "create_weight":7033, "shallow_advance":6887, "create_weight_count":1, "build_scorer":167377 } } ] } ], "rewrite_time":290412, "collector":[ { "name":"SimpleFieldCollector", "reason":"search_top_hits", "time_in_nanos":30033925274 } ] } ], "aggregations":[ ] } ] ``` Full right profile with search-after ```auto "searches":[ { "query":[ { "type":"BoostQuery", "description":"(ConstantScore(DocValuesFieldExistsQuery [field= __time__]))^0.0", "time_in_nanos":52342571431, "breakdown":{ "set_min_competitive_score_count":0, "match_count":0, "shallow_advance_count":0, "set_min_competitive_score":0, "next_doc":52342129926, "match":0, "next_doc_count":410870383, "score_count":0, "compute_max_score_count":0, "compute_max_score":0, "advance":30398, "advance_count":39, "score":0, "build_scorer_count":78, "create_weight":26799, "shallow_advance":0, "create_weight_count":1, "build_scorer":384308 }, "children":[ { "type":"DocValuesFieldExistsQuery", "description":"DocValuesFieldExistsQuery [field= __time__]", "time_in_nanos":20595411270, "breakdown":{ "set_min_competitive_score_count":0, "match_count":0, "shallow_advance_count":0, "set_min_competitive_score":0, "next_doc":20595073961, "match":0, "next_doc_count":410870383, "score_count":0, "compute_max_score_count":0, "compute_max_score":0, "advance":20634, "advance_count":39, "score":0, "build_scorer_count":78, "create_weight":7160, "shallow_advance":0, "create_weight_count":1, "build_scorer":309515 } } ] } ], "rewrite_time":220755, "collector":[ { "name":"PagingFieldCollector", "reason":"search_top_hits", "time_in_nanos":36757066196 } ] } ], "aggregations":[ ] } ] ``` ---
**Author:** ![MarvinLiu](https://avatars.discourse-cdn.com/v4/letter/m/ccd318/32.png) [@MarvinLiu](https://discuss.elastic.co/u/MarvinLiu)\ **Post date:** [September 7, 2021, 7:16am UTC](https://discuss.elastic.co/t/search-performance-difference-between-from-size-and-search-after/283400/7 "2021-09-07T07:16:43Z")
Thanks very much! just to be sure, have you compared both searches after a warmup period **and** both searches when compared either include or both do not include the `profile` boolean set? **-------- I have tested more than 10 times, and the results are the same every time, which has nothing to do with warm up。** Also, have you compared the profile output of both (there is a tool in kibana dev-tools that makes the output a little more graphical and easier to read). **-------see the above reply** In general using search after should require much less memory when paginated deeper (on the coordinating node), but both operations have to execute the same search. **------both are the same search，with or without search\_after，search\_after context offset is 10.** ```auto { "query":{ "bool":{ "must":{ "match_all":{ } }, "filter":[ { "range":{ " __time__":{ "gte":1324958207, "lte":1724958207 } } } ] } }, "size":10, "sort":[ { " __time__":{ "order":"desc" } }, { " __key__":{ "order":"desc" } } ] //, // "search_after":[ // 1630594662000, // "6130e666-2-67e9e3-f5-1" // ] } ``` I also guess the query\_string in there is just an example, as otherwise you could go with a `match_all` query or none as well? **-----I have tried `match_all` query, same result.** Also, which Elasticsearch version are you on? **---7.10** ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 5, 2021, 7:17am UTC](https://discuss.elastic.co/t/search-performance-difference-between-from-size-and-search-after/283400/8 "2021-10-05T07:17:04Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.