# Why inverted index is not good at Aggregation?
**URL:** https://discuss.elastic.co/t/why-inverted-index-is-not-good-at-aggregation/196298
**Category:** Elasticsearch
**Created:** [August 22, 2019, 10:40am UTC](https://discuss.elastic.co/t/why-inverted-index-is-not-good-at-aggregation/196298 "2019-08-22T10:40:34Z")
**Posts on this page:** 13
**Page:** 1
### Author:  [@hackerwin7](https://discuss.elastic.co/u/hackerwin7)
#### Post date: [August 22, 2019, 10:40am UTC](https://discuss.elastic.co/t/why-inverted-index-is-not-good-at-aggregation/196298/1 "2019-08-22T10:40:34Z")
[https://www.elastic.co/guide/en/elasticsearch/reference/current/fielddata.html#fielddata](https://www.elastic.co/guide/en/elasticsearch/reference/current/fielddata.html#fielddata)
this section of docs say sort and aggregation is another pattern to search:
search with inverted index:
_Which documents contain this term_ it is easy to understand to search term -\> documents contains this term
sort and aggregate:
_What is the value of this field for **this** document?_ so, the inverted index is not good at this.
My question is why sort and aggregate need to answer this question?
Thinking about the case below:
Inverted index:
Terms | doc\_1 | doc\_2 | doc\_3
abc | x | |
bcd | | x |
cde | | | x
sort :Because Terms have been sorted, just scan the inverted index with rows and list the doc:
sort Terms result: doc\_1, doc\_2, doc\_3
Inverted index
Terms | doc\_1 | doc\_2 | doc\_3 |
abc | x | | x |
bcd | | x | |
cde | x | | x |
aggregate: scan the inverted index:
aggregate count Terms:
abc bucket: doc\_1, doc\_3 -\> 2
bcd bucket: doc\_2 -\> 1
cde bucket: doc\_1, doc\_3 -\> 2
In these cases , why sort and aggregate need to know the document id -\> terms mapping ?
---
### Author:  [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [August 22, 2019, 12:15pm UTC](https://discuss.elastic.co/t/why-inverted-index-is-not-good-at-aggregation/196298/2 "2019-08-22T12:15:03Z")
> [@hackerwin7](#):
>
> My question is why sort and aggregate need to answer this question?
You typically search on one field and sort/aggregate on a different one.
Eg search text field for “iPad” but sort or aggregate on price field.
---
### Author:  [@hackerwin7](https://discuss.elastic.co/u/hackerwin7)
#### Post date: [August 23, 2019, 7:46am UTC](https://discuss.elastic.co/t/why-inverted-index-is-not-good-at-aggregation/196298/3 "2019-08-23T07:46:48Z")
@Mark_Harwood
Thanks for your reply.
How about search this field and sort/aggregate this field?
if not search any field, just sort this text field or aggregate this text field , is it to say that sort/aggregate do not need to answer the doc id -\> terms question, just scan the text field inverted index to implement sort and aggregate?
---
### Author:  [@wangqinghuan](https://discuss.elastic.co/u/wangqinghuan)
#### Post date: [August 23, 2019, 8:14am UTC](https://discuss.elastic.co/t/why-inverted-index-is-not-good-at-aggregation/196298/4 "2019-08-23T08:14:32Z")
The inverted index is designed for searching rather than sorting.Even not search any field, Es will fetch all documents id from inverted index and sort documents by id-\>terms mapping(column store).
---
### Author:  [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [August 23, 2019, 8:20am UTC](https://discuss.elastic.co/t/why-inverted-index-is-not-good-at-aggregation/196298/5 "2019-08-23T08:20:04Z")
> [@hackerwin7](#):
>
> just scan the text field inverted index to implement sort and aggregate?
Technically, yes, the information you need is potentially already in the inverted index - however it does not give you the O(1) lookup of the doc values structure
---
### Author:  [@hackerwin7](https://discuss.elastic.co/u/hackerwin7)
#### Post date: [August 23, 2019, 8:21am UTC](https://discuss.elastic.co/t/why-inverted-index-is-not-good-at-aggregation/196298/6 "2019-08-23T08:21:22Z")
@wangqinghuan
In this special case, I think the "fetch all documents id from inverted index and sort documents by id-\>terms mapping" is not necessary?
---
### Author:  [@hackerwin7](https://discuss.elastic.co/u/hackerwin7)
#### Post date: [August 23, 2019, 8:31am UTC](https://discuss.elastic.co/t/why-inverted-index-is-not-good-at-aggregation/196298/7 "2019-08-23T08:31:16Z")
> [@Mark\_Harwood](#):
>
> however it does not give you the O(1) lookup of the doc values structure
@Mark_Harwood
Overall, in this special case, the sort and aggregation is O(n)
sort process:
scan per terms get doc id, because terms is sorted, so sort the docs by terms can easily loop the terms and print the associated doc\_id
aggregation:
same as sort, loop the terms and count of associated doc\_id, we can get count of docs for aggregation of this filed term
I think , in this case, it is better than inverted index -\> doc id -\> access doc values to get terms -\> bucket agg etc.
---
### Author:  [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [August 23, 2019, 8:34am UTC](https://discuss.elastic.co/t/why-inverted-index-is-not-good-at-aggregation/196298/8 "2019-08-23T08:34:08Z")
> [@hackerwin7](#):
>
> I think , in this case, it is better than inverted index
PRs with benchmarks always welcome 🙂
---
### Author:  [@wangqinghuan](https://discuss.elastic.co/u/wangqinghuan)
#### Post date: [August 23, 2019, 8:47am UTC](https://discuss.elastic.co/t/why-inverted-index-is-not-good-at-aggregation/196298/9 "2019-08-23T08:47:44Z")
Each index contains multiple segments,where a segment is an inverted index. In your special case, you can sort documents by loop sorted term within one inverted index, though, how do you sort documents across mutiple segments?
---
### Author:  [@hackerwin7](https://discuss.elastic.co/u/hackerwin7)
#### Post date: [August 23, 2019, 8:59am UTC](https://discuss.elastic.co/t/why-inverted-index-is-not-good-at-aggregation/196298/10 "2019-08-23T08:59:29Z")
@wangqinghuan
My context is in single one inverted index. about across multiple segments sort or aggregate, what is the elasticsearch's solution to this?
In my special case, just **merge** the two sorted results into fixed one result
---
### Author:  [@Mikhail\_Khludnev](https://discuss.elastic.co/u/Mikhail_Khludnev)
#### Post date: [August 23, 2019, 11:01am UTC](https://discuss.elastic.co/t/why-inverted-index-is-not-good-at-aggregation/196298/11 "2019-08-23T11:01:14Z")
There are other search engines which aggregate terms through inverted index see slide 42 of [https://www.slideshare.net/lucenerevolution/seeley-solr-facetseurocon2011](https://www.slideshare.net/lucenerevolution/seeley-solr-facetseurocon2011) Although it gains only when number of terms and segments is small.
Still not getting for to sort with inv index.
---
### Author:  [@hackerwin7](https://discuss.elastic.co/u/hackerwin7)
#### Post date: [August 23, 2019, 11:21am UTC](https://discuss.elastic.co/t/why-inverted-index-is-not-good-at-aggregation/196298/12 "2019-08-23T11:21:56Z")
@Mikhail_Khludnev
Fantastic! Thanks your sharing resource.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [September 20, 2019, 11:21am UTC](https://discuss.elastic.co/t/why-inverted-index-is-not-good-at-aggregation/196298/13 "2019-09-20T11:21:57Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.