# Slow terms aggregation speed on ~130M documents **URL:** \ **Category:** Elasticsearch\ **Created:** [February 7, 2018, 6:32am UTC](https://discuss.elastic.co/t/slow-terms-aggregation-speed-on-130m-documents/118759 "2018-02-07T06:32:11Z")\ **Posts on this page:** 15\ **Page:** 2
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [February 8, 2018, 7:47am UTC](https://discuss.elastic.co/t/slow-terms-aggregation-speed-on-130m-documents/118759/22 "2018-02-08T07:47:01Z")
So you are using `doc_values` here. Which means that the OS Filesystem cache will be mostly used in that case. It can obviously take some time to read the data the first time but then you will most likely benefit from it. ---
**Author:** ![jdgenio](https://avatars.discourse-cdn.com/v4/letter/j/b9bd4f/32.png) [@jdgenio](https://discuss.elastic.co/u/jdgenio)\ **Post date:** [February 8, 2018, 7:56am UTC](https://discuss.elastic.co/t/slow-terms-aggregation-speed-on-130m-documents/118759/23 "2018-02-08T07:56:09Z")
If I'm not mistaken, OS level caching is what slows down the search on the first try? So in a way, we are sacrificing the first search speed for the performance of the subsequent searches? If that's the case, then I agree that it's a good tradeoff. Back to the main question, will preloading fielddata negate this first load problem or is setting up a CRON job the only viable solution? ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [February 8, 2018, 8:10am UTC](https://discuss.elastic.co/t/slow-terms-aggregation-speed-on-130m-documents/118759/24 "2018-02-08T08:10:10Z")
> [@jdgenio](#): > > If I'm not mistaken, OS level caching is what slows down the search on the first try? So in a way, we are sacrificing the first search speed for the performance of the subsequent searches? If that's the case, then I agree that it's a good tradeoff. Yes. > [@jdgenio](#): > > will preloading fielddata negate this first load problem or is setting up a CRON job the only viable solution? Again. You are not using [fielddata](https://www.elastic.co/guide/en/elasticsearch/reference/6.2/modules-fielddata.html) here right? If your question is "Will running some queries before the user be helpful?" the answer is yes. ---
**Author:** ![jdgenio](https://avatars.discourse-cdn.com/v4/letter/j/b9bd4f/32.png) [@jdgenio](https://discuss.elastic.co/u/jdgenio)\ **Post date:** [February 8, 2018, 8:18am UTC](https://discuss.elastic.co/t/slow-terms-aggregation-speed-on-130m-documents/118759/25 "2018-02-08T08:18:23Z")
What I mean is that if we add: ``` "fielddata": { "loading": "eager" } ``` to the keywords we need suggestions on, would it help lessen the load times? e.g.: from ``` "Consignee":{ "properties":{ "Name":{ "type":"text", "fields":{ "untouched":{ "type":"keyword" } } } } } ``` to ``` "Consignee":{ "properties":{ "Name":{ "type":"text", "fields":{ "untouched":{ "type":"keyword" }, "fielddata": { "loading": "eager" } } } } } ``` ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [February 8, 2018, 8:31am UTC](https://discuss.elastic.co/t/slow-terms-aggregation-speed-on-130m-documents/118759/26 "2018-02-08T08:31:05Z")
You might to look at [https://www.elastic.co/guide/en/elasticsearch/reference/6.2/eager-global-ordinals.html](https://www.elastic.co/guide/en/elasticsearch/reference/6.2/eager-global-ordinals.html) This can slow down your indexing process but be faster at search time may be. I'm sure @mvg might have lot of other ideas... 😃 ---
**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\ **Post date:** [February 8, 2018, 8:47am UTC](https://discuss.elastic.co/t/slow-terms-aggregation-speed-on-130m-documents/118759/27 "2018-02-08T08:47:18Z")
If you have a small set of results try an execution hint of “map” [https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html#search-aggregations-bucket-terms-aggregation-execution-hint](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html#search-aggregations-bucket-terms-aggregation-execution-hint) ---
**Author:** ![jdgenio](https://avatars.discourse-cdn.com/v4/letter/j/b9bd4f/32.png) [@jdgenio](https://discuss.elastic.co/u/jdgenio)\ **Post date:** [February 8, 2018, 9:31am UTC](https://discuss.elastic.co/t/slow-terms-aggregation-speed-on-130m-documents/118759/28 "2018-02-08T09:31:46Z")
Hi Mark, I haven't done some proper testing with benchmarks yet but so far, it was faster for my queries. What is the reason for the speed increase and what are the downsides to it? Looking at the docs, it implies that high result queries will be affected by the map execution hint. However, some of my queries have around a million results with no visible degradation compared to the default global\_ordinals query. What is considered a high result query? Also, thank you @dadoonet. Talking to you was a great learning experience. ---
**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\ **Post date:** [February 8, 2018, 9:45am UTC](https://discuss.elastic.co/t/slow-terms-aggregation-speed-on-130m-documents/118759/29 "2018-02-08T09:45:14Z")
Global ordinals execution mode aggregates information using numbers that each represent unique strings in the index. This means the memory overhead for labelling buckets is smaller than the length of the strings they replace but there's an overhead in the conversion. `map` execution tags the buckets being aggregated directly with the indexed value which could be long in some cases (imagine URL strings). If your query matches a lot of docs that's a lot of strings to hold in memory before they are pruned to your top N selection. > [@jdgenio](#): > > it was faster for my queries. Can you say how much faster? ---
**Author:** ![jdgenio](https://avatars.discourse-cdn.com/v4/letter/j/b9bd4f/32.png) [@jdgenio](https://discuss.elastic.co/u/jdgenio)\ **Post date:** [February 8, 2018, 9:57am UTC](https://discuss.elastic.co/t/slow-terms-aggregation-speed-on-130m-documents/118759/30 "2018-02-08T09:57:29Z")
We close the ES servers after office hours so I can't make queries right now to verify but earlier, the normal searches were taking around a little higher than 4 seconds while the one with the map execution hint took around 3.5 seconds, but some of the lower result queries were taking 2 or less. I'd have to perform a proper benchmark tomorrow and get back to you. > [@Mark\_Harwood](#): > > Global ordinals execution mode aggregates information using numbers that each represent unique strings in the index. This means the memory overhead for labelling buckets is smaller than the length of the strings they replace but there's an overhead in the conversion. > > map execution tags the buckets being aggregated directly with the indexed value which could be long in some cases (imagine URL strings). If your query matches a lot of docs that's a lot of strings to hold in memory before they are pruned to your top N selection. That makes a lot of sense, thank you for the explanation. If I understand correctly, @dadoonet's recommendation would then negate the global ordinal conversion overhead since it would eagerly build the global 'hashtable'? ---
**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\ **Post date:** [February 8, 2018, 10:06am UTC](https://discuss.elastic.co/t/slow-terms-aggregation-speed-on-130m-documents/118759/31 "2018-02-08T10:06:02Z")
If you are only using `map` execution hint on the field there is no need for global ordinals to be built so any "eager" loading would be wasted effort. ---
**Author:** ![jdgenio](https://avatars.discourse-cdn.com/v4/letter/j/b9bd4f/32.png) [@jdgenio](https://discuss.elastic.co/u/jdgenio)\ **Post date:** [February 8, 2018, 10:08am UTC](https://discuss.elastic.co/t/slow-terms-aggregation-speed-on-130m-documents/118759/32 "2018-02-08T10:08:13Z")
Yes, but using global ordinals while preloading it will be the better choice rather than use the map execution hint? ---
**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\ **Post date:** [February 8, 2018, 10:15am UTC](https://discuss.elastic.co/t/slow-terms-aggregation-speed-on-130m-documents/118759/33 "2018-02-08T10:15:31Z")
> [@jdgenio](#): > > Yes, but using global ordinals while preloading it will be the better choice rather than use the map execution hint? On a static index otherwise you are adding cost to every `refresh` when new content is added. ---
**Author:** ![jdgenio](https://avatars.discourse-cdn.com/v4/letter/j/b9bd4f/32.png) [@jdgenio](https://discuss.elastic.co/u/jdgenio)\ **Post date:** [February 8, 2018, 10:48am UTC](https://discuss.elastic.co/t/slow-terms-aggregation-speed-on-130m-documents/118759/34 "2018-02-08T10:48:14Z")
I see, it is also indicated on the docs as well. We would like to prioritize search speed over indexing speed. Worst comes to worst, we could schedule the document updates to a single bulk request when there is low user activity so there would be no daily refresh. Thank you very much for the insights, I think you already covered most if not all of my concerns. It has been a pleasure talking to you guys. Best of luck to the ElasticSearch team. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [March 8, 2018, 10:48am UTC](https://discuss.elastic.co/t/slow-terms-aggregation-speed-on-130m-documents/118759/35 "2018-03-08T10:48:22Z")
Closed on March 8, 2018, 10:48am UTC This topic was automatically closed 28 days after the last reply. New replies are no longer allowed. ---
**Author:** ![Alex\_Marquardt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alex_marquardt/32/42925_2.png) [@Alex\_Marquardt](https://discuss.elastic.co/u/Alex_Marquardt)\ **Post date:** [May 10, 2019, 6:58pm UTC](https://discuss.elastic.co/t/slow-terms-aggregation-speed-on-130m-documents/118759/36 "2019-05-10T18:58:04Z")
If the field which you are aggregating on is high cardinality, then the slow response may be due to computation of global ordinals (as @Mark_Harwood mentioned above). For future reference, the following blog post gives an overview of the underlying cause of slowness and several approaches to address this issue. > **[Improving the performance of high-cardinality terms aggregations in...](https://www.elastic.co/blog/improving-the-performance-of-high-cardinality-terms-aggregations-in-elasticsearch)** > > Learn how to improve the performance of slow terms aggregations on high-cardinality fields. Don't let fields with a high level of uniqueness slow down your Elasticsearch queries. [Previous page](https://discuss.elastic.co/t/slow-terms-aggregation-speed-on-130m-documents/118759.md?page=1)