# Filtering while searching

**URL:** <https://discuss.elastic.co/t/filtering-while-searching/141912>\
**Category:** Elasticsearch\
**Created:** [July 27, 2018, 9:08am UTC](https://discuss.elastic.co/t/filtering-while-searching/141912 "2018-07-27T09:08:59Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Anton\_Garakh](https://avatars.discourse-cdn.com/v4/letter/a/5f8ce5/32.png) [@Anton\_Garakh](https://discuss.elastic.co/u/Anton_Garakh)\
**Post date:** [July 27, 2018, 9:08am UTC](https://discuss.elastic.co/t/filtering-while-searching/141912/1 "2018-07-27T09:08:59Z")

</div>

Hi all

Let's say I have documents of the following structure:

```
Product {
organization;
supplier;
name;
}

```

and there are billions of documents, a lot of organizations, and each organization has approximately 100000 products . and when searching for products in particular organization by name, It seems to me that it's beneficial to make ElasticSearch first filter out products by organization and then, having only 100000 products out of billions, search by name.

I tried to accomplish this the following way:

```
        {  
   "query":{  
      "bool":{  
         "must":[  
            {  
               "term":{  
                  "name":{  
                     "value":"test"
                  }
               }
            }
         ],
         "filter":[  
            {  
               "term":{  
                  "organization":"123"
               }
            }
         ]
      }
   }
}

```

but have got worse results in terms of performance that using this query:

```
{  
   "query":{  
      "bool":{  
         "must":[  
            {  
               "term":{  
                  "name":{  
                     "value":"test"
                  }
               }
            },
            {  
               "term":{  
                  "organization":{  
                     "value":"123"
                  }
               }
            }
         ]
      }
   }
}

```

Can anybody explain why? And any suggestions how to achieve my goal?

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [July 27, 2018, 9:26am UTC](https://discuss.elastic.co/t/filtering-while-searching/141912/2 "2018-07-27T09:26:51Z")

</div>

Filters can be cached because they just match or don't match.  
Queries (must) can't be cached because they are used to compute the score. And this is changing every time you are adding documents as you need to recompute everything.

So the very first run with a filter might be a bit slow (less than for a query though) and then super fast.

In short, if a part of the query is not supposed to modify the score, put it in a filter everytime...

If your goal is to always filter by companies, I'd encourage you using routing by organisation so you will also search within one single shard.

---

<div class="post-metadata">

**Author:** ![Anton\_Garakh](https://avatars.discourse-cdn.com/v4/letter/a/5f8ce5/32.png) [@Anton\_Garakh](https://discuss.elastic.co/u/Anton_Garakh)\
**Post date:** [July 27, 2018, 9:47am UTC](https://discuss.elastic.co/t/filtering-while-searching/141912/3 "2018-07-27T09:47:21Z")

</div>

What do you mean by modifying score? What I need is having results with score of 1.0 always.

If I understand you correctly the first query in question will 100% work better than the second one, won't it?  
Could you please give me a query which will perform better than the two I posted?

What do you mean by routing organization?

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [July 27, 2018, 9:59am UTC](https://discuss.elastic.co/t/filtering-while-searching/141912/4 "2018-07-27T09:59:24Z")

</div>

The first query is better.  
Even better if you don't care at all about score:

```auto
{  
   "query":{  
      "bool":{  
         "filter":[  
            {  
               "term":{  
                  "name":{  
                     "value":"test"
                  }
               }
            },
            {  
               "term":{  
                  "organization":"123"
               }
            }
         ]
      }
   }
}

```

> What do you mean by routing organization?

> **[\_routing field | Elasticsearch Guide \[8.11\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-routing-field.html)**

---

<div class="post-metadata">

**Author:** ![Anton\_Garakh](https://avatars.discourse-cdn.com/v4/letter/a/5f8ce5/32.png) [@Anton\_Garakh](https://discuss.elastic.co/u/Anton_Garakh)\
**Post date:** [July 27, 2018, 10:15am UTC](https://discuss.elastic.co/t/filtering-while-searching/141912/5 "2018-07-27T10:15:54Z")

</div>

Thanks for a link

As for scoring, like I said, I need score of 1.0. As far as I know, filter can return results even when scoring is less than 1.0. So I bet filtering on name, which can return not 100% matching results, is not the case for me

By the way, can score be different for searching on field that 100% have only one token?

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [July 27, 2018, 10:50am UTC](https://discuss.elastic.co/t/filtering-while-searching/141912/6 "2018-07-27T10:50:13Z")

</div>

> I need score of 1.0

Not sure if we are speaking about the same thing.

```auto
GET /_search

```

Gives a score of 1.0.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 24, 2018, 10:50am UTC](https://discuss.elastic.co/t/filtering-while-searching/141912/7 "2018-08-24T10:50:38Z")

</div>

Closed on August 24, 2018, 10:50am UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.