# About distributed IDF in ES

**URL:** <https://discuss.elastic.co/t/about-distributed-idf-in-es/3371>\
**Category:** Elasticsearch\
**Created:** [September 27, 2010, 1:57pm UTC](https://discuss.elastic.co/t/about-distributed-idf-in-es/3371 "2010-09-27T13:57:53Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![thiago](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thiago/32/32096_2.png) [@thiago](https://discuss.elastic.co/u/thiago)\
**Post date:** [September 27, 2010, 1:57pm UTC](https://discuss.elastic.co/t/about-distributed-idf-in-es/3371/1 "2010-09-27T13:57:53Z")

</div>

Hi Shay,

```
 One of the main limitation of building a distributed Solr was that

```

there would be no distributed IDF, and so, the issue SOLR-1632 was raised to  
address this limitation. How is this issue handled in ES?

Regards,  
Thiago Souza

---

<div class="post-metadata">

**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\
**Post date:** [September 27, 2010, 2:01pm UTC](https://discuss.elastic.co/t/about-distributed-idf-in-es/3371/2 "2010-09-27T14:01:13Z")

</div>

Don't know about Solr in this aspect, but here is what elasticsearch has to  
offer:  
[http://www.elasticsearch.com/docs/elasticsearch/rest\_api/search/search\_type/](http://www.elasticsearch.com/docs/elasticsearch/rest_api/search/search_type/)(check  
the dfs\_... options).

-shay.banon

On Mon, Sep 27, 2010 at 3:57 PM, Thiago Souza [tcostasouza@gmail.com](mailto:tcostasouza@gmail.com) wrote:

> Hi Shay,
> 
> ```
> One of the main limitation of building a distributed Solr was that
> 
> ```
> 
> there would be no distributed IDF, and so, the issue SOLR-1632 was raised to  
> address this limitation. How is this issue handled in ES?
> 
> Regards,  
> Thiago Souza

---

<div class="post-metadata">

**Author:** ![jminard](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jminard/32/1848_2.png) [@jminard](https://discuss.elastic.co/u/jminard)\
**Post date:** [November 8, 2010, 3:01am UTC](https://discuss.elastic.co/t/about-distributed-idf-in-es/3371/3 "2010-11-08T03:01:28Z")

</div>

## So, a double query to compute the adjustments to normalize IDF. It is possible to generate an IDF specific lookup (i.e. index) that is used locally in master nodes rather than fanning out a query. I guess it depends on the latency of this 2-pass approach whether it is worth it to go for something that manages an index-time list vs. doing it at query time. Endeca for example has an aggregator index which contains global/distributed IDF information for those master processes that start the query chain.

View this message in context: [http://elasticsearch-users.115913.n3.nabble.com/About-distributed-IDF-in-ES-tp1589207p1860688.html](http://elasticsearch-users.115913.n3.nabble.com/About-distributed-IDF-in-ES-tp1589207p1860688.html)  
Sent from the ElasticSearch Users mailing list archive at [Nabble.com](http://Nabble.com).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:16am UTC](https://discuss.elastic.co/t/about-distributed-idf-in-es/3371/4 "2017-07-06T04:16:57Z")

</div>

Closed on July 6, 2017, 4:16am UTC