# Using the aggregation Framework using a large set of doc IDs as query? ( + bypassing the scoring part)
**URL:** https://discuss.elastic.co/t/using-the-aggregation-framework-using-a-large-set-of-doc-ids-as-query-bypassing-the-scoring-part/17170
**Category:** Elasticsearch
**Created:** [April 24, 2014, 9:19am UTC](https://discuss.elastic.co/t/using-the-aggregation-framework-using-a-large-set-of-doc-ids-as-query-bypassing-the-scoring-part/17170 "2014-04-24T09:19:23Z")
**Posts on this page:** 5
**Page:** 1
### Author:  [@nicom](https://discuss.elastic.co/u/nicom)
#### Post date: [April 24, 2014, 9:19am UTC](https://discuss.elastic.co/t/using-the-aggregation-framework-using-a-large-set-of-doc-ids-as-query-bypassing-the-scoring-part/17170/1 "2014-04-24T09:19:23Z")
Hi guys,
I use my own framework and get already the top N results from a previous
processing.
I would like to use the aggregation framework of ES to use facets & co
features on such results.
I previously indexed my documents in ES.
What ES query should I do to avoid the scoring process and process only the
aggregation facets and co features using the IDs a set of documents as
query _knowing that N could be large (N = 1K)_?
JAVA API
Thanks
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/0b6b4e52-11fe-4914-b1bb-ed4b69421c08%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0b6b4e52-11fe-4914-b1bb-ed4b69421c08%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@radu\_gheorghe](https://discuss.elastic.co/u/radu_gheorghe)
#### Post date: [April 24, 2014, 11:40am UTC](https://discuss.elastic.co/t/using-the-aggregation-framework-using-a-large-set-of-doc-ids-as-query-bypassing-the-scoring-part/17170/2 "2014-04-24T11:40:22Z")
Hello,
One way to do it would be to store all those IDs in an Elasticsearch
document. Then, you can use the terms filter with the terms lookup
mechanism to have ES fetch all the terms for you:
> **[Elastic — The Search AI Company](https://www.elastic.co)**
>
> Power insights and outcomes with The Elastic Search AI Platform. See into your data and find answers that matter with enterprise solutions designed to help you accelerate time to insight. Try Elastic ...
As you can see there, you have quite a lot of options for caching.
## Best regards, Radu
Performance Monitoring \* Log Analytics \* Search Analytics
Solr & Elasticsearch Support \* [http://sematext.com/](http://sematext.com/)
On Thu, Apr 24, 2014 at 12:19 PM, NM [n.maisonneuve@gmail.com](mailto:n.maisonneuve@gmail.com) wrote:
> Hi guys,
>
> I use my own framework and get already the top N results from a previous
> processing.
>
> I would like to use the aggregation framework of ES to use facets & co
> features on such results.
>
> I previously indexed my documents in ES.
>
> What ES query should I do to avoid the scoring process and process only
> the aggregation facets and co features using the IDs a set of documents as
> query _knowing that N could be large (N = 1K)_?
>
> JAVA API
>
> Thanks
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> To view this discussion on the web visit
> [https://groups.google.com/d/msgid/elasticsearch/0b6b4e52-11fe-4914-b1bb-ed4b69421c08%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0b6b4e52-11fe-4914-b1bb-ed4b69421c08%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/0b6b4e52-11fe-4914-b1bb-ed4b69421c08%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/0b6b4e52-11fe-4914-b1bb-ed4b69421c08%40googlegroups.com?utm_medium=email&utm_source=footer)
> .
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAHXA0\_3cZynh5WmcWFEH0oQFzLWNg-b600Wbu%3DEsOyFiPZBwkA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAHXA0_3cZynh5WmcWFEH0oQFzLWNg-b600Wbu%3DEsOyFiPZBwkA%40mail.gmail.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@nicom](https://discuss.elastic.co/u/nicom)
#### Post date: [April 24, 2014, 3:25pm UTC](https://discuss.elastic.co/t/using-the-aggregation-framework-using-a-large-set-of-doc-ids-as-query-bypassing-the-scoring-part/17170/3 "2014-04-24T15:25:43Z")
thanks Radu,
to be sure to understand:
I have a query from a user, run a process A returning a list of IDs
specific of the query and would like to use ES to enrich these ID with
aggregated info coming from the related (and already indexed) documents
so the list of IDs from the results are a prior unknown / depends on the
query of the user.
to use only the aggregation framework, you propose then for each query, to
first index the results of the process A (list of ID) as a lookup document.
and then after query ES using a term filter + lookup mechanism.
Is that right?
Le jeudi 24 avril 2014 13:40:22 UTC+2, Radu Gheorghe a écrit :
> Hello,
>
> One way to do it would be to store all those IDs in an Elasticsearch
> document. Then, you can use the terms filter with the terms lookup
> mechanism to have ES fetch all the terms for you:
>
> [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-terms-filter.html#_terms_lookup_mechanism)
>
> As you can see there, you have quite a lot of options for caching.
>
> ## Best regards, Radu
>
> Performance Monitoring \* Log Analytics \* Search Analytics
> Solr & Elasticsearch Support \* [http://sematext.com/](http://sematext.com/)
>
> On Thu, Apr 24, 2014 at 12:19 PM, NM \<[n.mais...@gmail.com](mailto:n.mais...@gmail.com) \\>wrote:
>
> > Hi guys,
> >
> > I use my own framework and get already the top N results from a previous
> > processing.
> >
> > I would like to use the aggregation framework of ES to use facets & co
> > features on such results.
> >
> > I previously indexed my documents in ES.
> >
> > What ES query should I do to avoid the scoring process and process only
> > the aggregation facets and co features using the IDs a set of documents as
> > query _knowing that N could be large (N = 1K)_?
> >
> > JAVA API
> >
> > Thanks
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "elasticsearch" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \.
> > To view this discussion on the web visit
> > [https://groups.google.com/d/msgid/elasticsearch/0b6b4e52-11fe-4914-b1bb-ed4b69421c08%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0b6b4e52-11fe-4914-b1bb-ed4b69421c08%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/0b6b4e52-11fe-4914-b1bb-ed4b69421c08%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/0b6b4e52-11fe-4914-b1bb-ed4b69421c08%40googlegroups.com?utm_medium=email&utm_source=footer)
> > .
> > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/913a7d06-ec51-4d4d-9d7a-e95750015c4b%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/913a7d06-ec51-4d4d-9d7a-e95750015c4b%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@radu\_gheorghe](https://discuss.elastic.co/u/radu_gheorghe)
#### Post date: [April 25, 2014, 2:24pm UTC](https://discuss.elastic.co/t/using-the-aggregation-framework-using-a-large-set-of-doc-ids-as-query-bypassing-the-scoring-part/17170/4 "2014-04-25T14:24:26Z")
Hi,
Yes, that's what I said, but I didn't know you wanted to use ES to enrich
the results only.
Plus, since you have the IDs already, you might use a huge multi-get
request:
[Elastic — The Search AI Company | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-multi-get.html)[http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-multi-get.html#docs-multi-get](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-multi-get.html#docs-multi-get)
## Best regards, Radu
Performance Monitoring \* Log Analytics \* Search Analytics
Solr & Elasticsearch Support \* [http://sematext.com/](http://sematext.com/)
On Thu, Apr 24, 2014 at 6:25 PM, NM [n.maisonneuve@gmail.com](mailto:n.maisonneuve@gmail.com) wrote:
> thanks Radu,
>
> to be sure to understand:
>
> I have a query from a user, run a process A returning a list of IDs
> specific of the query and would like to use ES to enrich these ID with
> aggregated info coming from the related (and already indexed) documents
>
> so the list of IDs from the results are a prior unknown / depends on the
> query of the user.
>
> to use only the aggregation framework, you propose then for each query, to
> first index the results of the process A (list of ID) as a lookup document.
> and then after query ES using a term filter + lookup mechanism.
>
> Is that right?
>
> Le jeudi 24 avril 2014 13:40:22 UTC+2, Radu Gheorghe a écrit :
>
> > Hello,
> >
> > One way to do it would be to store all those IDs in an Elasticsearch
> > document. Then, you can use the terms filter with the terms lookup
> > mechanism to have ES fetch all the terms for you:
> > [Elastic — The Search AI Company | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/)
> > reference/current/query-dsl-terms-filter.html#\_terms\_lookup\_mechanism
> >
> > As you can see there, you have quite a lot of options for caching.
> >
> > ## Best regards, Radu
> >
> > Performance Monitoring \* Log Analytics \* Search Analytics
> > Solr & Elasticsearch Support \* [http://sematext.com/](http://sematext.com/)
> >
> > On Thu, Apr 24, 2014 at 12:19 PM, NM [n.mais...@gmail.com](mailto:n.mais...@gmail.com) wrote:
> >
> > > Hi guys,
> > >
> > > I use my own framework and get already the top N results from a
> > > previous processing.
> > >
> > > I would like to use the aggregation framework of ES to use facets & co
> > > features on such results.
> > >
> > > I previously indexed my documents in ES.
> > >
> > > What ES query should I do to avoid the scoring process and process only
> > > the aggregation facets and co features using the IDs a set of documents as
> > > query _knowing that N could be large (N = 1K)_?
> > >
> > > JAVA API
> > >
> > > Thanks
> > >
> > > --
> > > You received this message because you are subscribed to the Google
> > > Groups "elasticsearch" group.
> > > To unsubscribe from this group and stop receiving emails from it, send
> > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > >
> > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/)
> > > msgid/elasticsearch/0b6b4e52-11fe-4914-b1bb-ed4b69421c08%
> > > [40googlegroups.com](http://40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/0b6b4e52-11fe-4914-b1bb-ed4b69421c08%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/0b6b4e52-11fe-4914-b1bb-ed4b69421c08%40googlegroups.com?utm_medium=email&utm_source=footer)
> > > .
> > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "elasticsearch" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> > To view this discussion on the web visit
> > [https://groups.google.com/d/msgid/elasticsearch/913a7d06-ec51-4d4d-9d7a-e95750015c4b%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/913a7d06-ec51-4d4d-9d7a-e95750015c4b%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/913a7d06-ec51-4d4d-9d7a-e95750015c4b%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/913a7d06-ec51-4d4d-9d7a-e95750015c4b%40googlegroups.com?utm_medium=email&utm_source=footer)
> > .
>
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAHXA0\_0mroJemTpy4wygCOzBLsBXc1hWBYMdVLJC%3D0vinpmCdA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAHXA0_0mroJemTpy4wygCOzBLsBXc1hWBYMdVLJC%3D0vinpmCdA%40mail.gmail.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 1:33am UTC](https://discuss.elastic.co/t/using-the-aggregation-framework-using-a-large-set-of-doc-ids-as-query-bypassing-the-scoring-part/17170/5 "2017-07-06T01:33:31Z")