How does Elasticsearch use Lucene for aggregations?

Elastic Stack Elasticsearch
1

Hello,

I could not find this information while reading either Elasticsearch Definitive Guide or Elasticsearch Reference, which led me to trying out the Community.

I would like to learn how does Elasticsearch perform aggregations - particularly calculating top N most frequently occurring terms - with the knowledge that its segments are Lucene indexes. Does it make use of Lucene Facets API when creating Lucene documents? What Lucene queries are performed by Elasticsearch when calculating aggregations?

I have a local Elasticsearch node running with debugger attached using IntelliJ, so any hints on where to look in the source code would be beneficial. All kind of explanation is also welcome.

2

Lucene does have join queries since version 4.x [1] and Elasticsearch
abstracts the details of using those queries with the various joining
queries such as Nested and Parent/Child.

Aggregations are done using custom code within Elasticsearch and does not
use the Facets API. I believe that Solr, like Elasticsearch, also does not
use the Facets API, but my knowledge of Solr is several versions old. I
have not explored the aggregations code in detail, but I am assuming the
Elasticsearch leverages the Lucene doc_values/fielddata APIs and does not
use queries to calculate the aggregations.

[1]
https://lucene.apache.org/core/6_0_0/join/org/apache/lucene/search/join/package-summary.html

1 Like
3

Thanks Ivan!

I have not heard yet of the Lucene join queries, which is an interesting topic for me to investigate on its own.
From what I have recently found there are no taxonomy directories created for Lucene indexes, which sounds like a proof that Facets API is not used by Elastic.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.