# Conditionally add document only if percolator matches a query

**URL:** https://discuss.elastic.co/t/conditionally-add-document-only-if-percolator-matches-a-query/8498
**Category:** Elasticsearch
**Created:** [July 24, 2012, 5:18am UTC](https://discuss.elastic.co/t/conditionally-add-document-only-if-percolator-matches-a-query/8498 "2012-07-24T05:18:46Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![tiftif](https://avatars.discourse-cdn.com/v4/letter/t/90ced4/32.png) [@tiftif](https://discuss.elastic.co/u/tiftif)
#### Post date: [July 24, 2012, 5:18am UTC](https://discuss.elastic.co/t/conditionally-add-document-only-if-percolator-matches-a-query/8498/1 "2012-07-24T05:18:46Z")

</div>

Our current setup looks like this:

1. We have a bunch of queries defined in ES (just a few)
2. We have an incoming data stream.
3. We percolate (and thus add) each document into ES.
4. We check the resulting queries from percolation. If no queries matched,  
we delete the added document.

The problem is that we are constantly adding and removing documents into  
ES. Just running ES for 2 days runs out of memory for some reason.

Is there a way to conditionally add a document only if it matches one or  
more queries after percolation?

---

<div class="post-metadata">

### Author: ![tiftif](https://avatars.discourse-cdn.com/v4/letter/t/90ced4/32.png) [@tiftif](https://discuss.elastic.co/u/tiftif)
#### Post date: [July 24, 2012, 5:26am UTC](https://discuss.elastic.co/t/conditionally-add-document-only-if-percolator-matches-a-query/8498/2 "2012-07-24T05:26:24Z")

</div>

Btw, to be more clear, I understand we can do this individually for each  
document (i.e., only percolate each document, then add each document  
selectively). But is there a bulk API to do this?

On Tuesday, July 24, 2012 1:18:46 AM UTC-4, tiftif wrote:

> Our current setup looks like this:
> 
> 1. We have a bunch of queries defined in ES (just a few)
> 2. We have an incoming data stream.
> 3. We percolate (and thus add) each document into ES.
> 4. We check the resulting queries from percolation. If no queries matched,  
> we delete the added document.
> 
> The problem is that we are constantly adding and removing documents into  
> ES. Just running ES for 2 days runs out of memory for some reason.
> 
> Is there a way to conditionally add a document only if it matches one or  
> more queries after percolation?

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [July 24, 2012, 7:07am UTC](https://discuss.elastic.co/t/conditionally-add-document-only-if-percolator-matches-a-query/8498/3 "2012-07-24T07:07:14Z")

</div>

Just wondering what is your setup ?

Do you have multiple nodes? Only one embedded instance in your own project?

Where do you see OOM? On the Elasticsearch nodes?

How do you percolate? With a Node Client or with a Transport Client ?

David

De : [elasticsearch@googlegroups.com](mailto:elasticsearch@googlegroups.com) [[mailto:elasticsearch@googlegroups.com](mailto:elasticsearch@googlegroups.com)] De la part de tiftif  
Envoyé : mardi 24 juillet 2012 07:26  
À : [elasticsearch@googlegroups.com](mailto:elasticsearch@googlegroups.com)  
Objet : Re: Conditionally add document only if percolator matches a query

Btw, to be more clear, I understand we can do this individually for each document (i.e., only percolate each document, then add each document selectively). But is there a bulk API to do this?

On Tuesday, July 24, 2012 1:18:46 AM UTC-4, tiftif wrote:

Our current setup looks like this:

1. We have a bunch of queries defined in ES (just a few)

2. We have an incoming data stream.

3. We percolate (and thus add) each document into ES.

4. We check the resulting queries from percolation. If no queries matched, we delete the added document.

The problem is that we are constantly adding and removing documents into ES. Just running ES for 2 days runs out of memory for some reason.

Is there a way to conditionally add a document only if it matches one or more queries after percolation?

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 3:19am UTC](https://discuss.elastic.co/t/conditionally-add-document-only-if-percolator-matches-a-query/8498/4 "2017-07-06T03:19:21Z")

</div>


