# Filter percolators before percolation query

**URL:** <https://discuss.elastic.co/t/filter-percolators-before-percolation-query/76050>\
**Category:** Elasticsearch\
**Created:** [February 22, 2017, 2:18pm UTC](https://discuss.elastic.co/t/filter-percolators-before-percolation-query/76050 "2017-02-22T14:18:28Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![frankkoornstra](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/frankkoornstra/32/6780_2.png) [@frankkoornstra](https://discuss.elastic.co/u/frankkoornstra)\
**Post date:** [February 22, 2017, 2:18pm UTC](https://discuss.elastic.co/t/filter-percolators-before-percolation-query/76050/1 "2017-02-22T14:18:28Z")

</div>

Percolation changed a bit coming from 2.4 to 5.2. In 2.4 we had the opportunity to filter the percolators that were used before the percolation actually took place (see _Additional supported query string options_ under [Percolator API in 2.4](https://www.elastic.co/guide/en/elasticsearch/reference/2.4/search-percolate.html#_percolate_api)).

In the 5.2 [Percolator Query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-percolate-query.html) documentation I can't seem to find a way of filtering the percolators first.

I think it's probably done with some clever combination of other filter but I can't seem to find out how to do it. Any tips?

---

<div class="post-metadata">

**Author:** ![javanna](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/javanna/32/4698_2.png) [@javanna](https://discuss.elastic.co/u/javanna)\
**Post date:** [February 22, 2017, 3:38pm UTC](https://discuss.elastic.co/t/filter-percolators-before-percolation-query/76050/2 "2017-02-22T15:38:59Z")

</div>

HI Frank,  
given that percolate is now a query, you can just use the query as part of a bool query which also has a filter clause. That one will filter the documents being percolated.

Also, we now index terms extracted from most of the queries, so we perform pre-filtering internally even if you don't provide any.

Cheers  
Luca

---

<div class="post-metadata">

**Author:** ![frankkoornstra](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/frankkoornstra/32/6780_2.png) [@frankkoornstra](https://discuss.elastic.co/u/frankkoornstra)\
**Post date:** [February 22, 2017, 3:47pm UTC](https://discuss.elastic.co/t/filter-percolators-before-percolation-query/76050/3 "2017-02-22T15:47:17Z")

</div>

Thanks for your quick reply but what I was looking for was a way to filter the percolators that are applied _not_ the documents being percolated. The link to the docs section on _Additional supported query string options_ at the [Percolator API in 2.4](https://www.elastic.co/guide/en/elasticsearch/reference/2.4/search-percolate.html#_percolate_api) shows that nicely

---

<div class="post-metadata">

**Author:** ![frankkoornstra](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/frankkoornstra/32/6780_2.png) [@frankkoornstra](https://discuss.elastic.co/u/frankkoornstra)\
**Post date:** [February 27, 2017, 3:16pm UTC](https://discuss.elastic.co/t/filter-percolators-before-percolation-query/76050/4 "2017-02-27T15:16:14Z")

</div>

This was existing functionality in ES 2.4 and I can't imagine that it was thrown out because it's very useful to limit the amount of resources needed to percolate. Anyone else?

---

<div class="post-metadata">

**Author:** ![frankkoornstra](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/frankkoornstra/32/6780_2.png) [@frankkoornstra](https://discuss.elastic.co/u/frankkoornstra)\
**Post date:** [March 2, 2017, 11:56am UTC](https://discuss.elastic.co/t/filter-percolators-before-percolation-query/76050/5 "2017-03-02T11:56:04Z")

</div>

To clarify, an example query would be

```auto
{
  "doc": {
    "created_at": "2010-10-10T00:00:00",
    "message": "some text"
  },
  "filter": {
    "term": {
      "percolator_group_name": "category"
    }
  }
}

```

that way we could percolate only categories (and not all other percolate groups like, for example tags). The percolator group name would be just a non-analyzed string (in 2.4) field in the `.percolator` index. I can still put in the extra field in 5.x (as `keyword` type) but I can't seem to find the filter option as shown in the query above to filter for only specific _percolators_

---

<div class="post-metadata">

**Author:** ![joejoe](https://avatars.discourse-cdn.com/v4/letter/j/f17d59/32.png) [@joejoe](https://discuss.elastic.co/u/joejoe)\
**Post date:** [March 16, 2017, 5:08pm UTC](https://discuss.elastic.co/t/filter-percolators-before-percolation-query/76050/6 "2017-03-16T17:08:36Z")

</div>

Just upgraded to 5.2 and I knew that I had to migrate to percolate query but just as you, I couldn't find a way to filter the percolators by meta data (this is practically a showstopper for us).

Did you find a solution?

Thanks,  
joe

---

<div class="post-metadata">

**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\
**Post date:** [March 16, 2017, 6:58pm UTC](https://discuss.elastic.co/t/filter-percolators-before-percolation-query/76050/7 "2017-03-16T18:58:26Z")

</div>

I don't know percolation very well but I expect something like this will work:

Modify the doc:

```auto
{
    "created_at": "2010-10-10T00:00:00",
    "message": "some text",
    "percolator_group_name": "some name"
}

```

Modify the percolator:

```auto
{
   "bool": {
      "must": [
         {"term": {"percolator_group_name": "some name"}},
         {your original percolator}
      ]
   }
}

```

With the term extraction this _ought_ to filter the attempted percolators. You don't get to use the same document, but it ought to work.

---

<div class="post-metadata">

**Author:** ![mvg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mvg/32/98890_2.png) [@mvg](https://discuss.elastic.co/u/mvg)\
**Post date:** [March 16, 2017, 7:15pm UTC](https://discuss.elastic.co/t/filter-percolators-before-percolation-query/76050/8 "2017-03-16T19:15:43Z")

</div>

The example that @nik9000 has shared is the way to prefilter percolator queries based on their metadata before percolation. This example in 5.x results in the same behavior with the search api as @frankkoornstra example in 2.x with the percolate api.

---

<div class="post-metadata">

**Author:** ![joejoe](https://avatars.discourse-cdn.com/v4/letter/j/f17d59/32.png) [@joejoe](https://discuss.elastic.co/u/joejoe)\
**Post date:** [March 17, 2017, 8:27am UTC](https://discuss.elastic.co/t/filter-percolators-before-percolation-query/76050/9 "2017-03-17T08:27:49Z")

</div>

Thanks @nik9000 and @mvg , that would technically work.

My use case: the end users define the percolator queries and on save I attach meta data to it, like company (this will also be on the document), consern/corporate (not on the doc), region (not on the doc), the user creating the percolator (not on the doc) etc. This way the user can maintain the query and the application the meta data - very straight forward and elegant.

The suggested 5.x way requires me altering both the user's query (but show only the user's part to the user in the UI editor) and the document - doable but feels forced.

Any chance of re-implementing meta data on percolator? 🙂

---

<div class="post-metadata">

**Author:** ![mvg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mvg/32/98890_2.png) [@mvg](https://discuss.elastic.co/u/mvg)\
**Post date:** [March 17, 2017, 9:57am UTC](https://discuss.elastic.co/t/filter-percolators-before-percolation-query/76050/10 "2017-03-17T09:57:46Z")

</div>

@Joejoe The use case you describe doesn't require you to alter the user's query. In fact adding metadata to a percolator query remained the same between 2.x and 5.x.

The main difference here is that a percolate time you use the percolate query in the search api. The percolate query doesn't contain the users query, but the document you like to percolate. If in addition you also like to filter percolator queries on previously attached metadata then you change the search request body that have a bool query with two or more must clauses. One clause for the percolate query and other must clauses for each metadata property you like to filter on. Similar how you would define filters in the 2.x percolate api.

Can you share rest / curl examples of steps you think you need to do? Then I get a better idea what you mean. I think we are somehow misunderstanding each other.

---

<div class="post-metadata">

**Author:** ![joejoe](https://avatars.discourse-cdn.com/v4/letter/j/f17d59/32.png) [@joejoe](https://discuss.elastic.co/u/joejoe)\
**Post date:** [March 17, 2017, 10:46am UTC](https://discuss.elastic.co/t/filter-percolators-before-percolation-query/76050/11 "2017-03-17T10:46:12Z")

</div>

@mvg thanks for following up. I'm travelling now but I'll get back with concrete example on monday.

---

<div class="post-metadata">

**Author:** ![frankkoornstra](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/frankkoornstra/32/6780_2.png) [@frankkoornstra](https://discuss.elastic.co/u/frankkoornstra)\
**Post date:** [March 20, 2017, 11:06am UTC](https://discuss.elastic.co/t/filter-percolators-before-percolation-query/76050/12 "2017-03-20T11:06:00Z")

</div>

@nik9000 @mvg thanks! Now that I see it, it seems so straightforward 😃

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 17, 2017, 11:06am UTC](https://discuss.elastic.co/t/filter-percolators-before-percolation-query/76050/13 "2017-04-17T11:06:08Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.