Can I percolate a document by just supplying an ID?

Elastic Stack Elasticsearch
1

Hi all,
Probably a newbie question, but I can't seem to find an example of this
anywhere.

If I create a document like so:
curl -XPUT http://localhost:9200/test/testing/1 -d '{
"field1" : "value1"
}'

If I run through the example for creating a percolator like so:

curl -XPUT localhost:9200/_percolator/test/kuku -d '{
"query" : {
"term" : {
"field1" : "value1"
}
}
}'

The documentation says I can percolate a document to see what queries match
by doing this:

curl -XGET localhost:9200/test/type1/_percolate -d '{
"doc" : {
"field1" : "value1"
}
}'

But what if I wanted to percolate the document by passing in it's ID
instead of the entire document? Is this possible? or is it assumed that
percolating is only ever done when a document is created or updated?

Seems like maybe I'm missing something

--

2

Hi Allan,

I don't know a direct answer to your question, but a way to achieve
the same thing is to pass the ID(s) you want through an IDs query[0],
and wrap that up in a bool query[1] along with the other query(es)
that you register as percolator(s).

[0] Elasticsearch Platform — Find real-time answers at scale | Elastic
[1] Elasticsearch Platform — Find real-time answers at scale | Elastic

Best regards,
Radu

http://sematext.com/ -- Elasticsearch -- Solr -- Lucene

On Tue, Nov 20, 2012 at 1:08 AM, Allan Mercado amercado@darelle.com wrote:

Hi all,
Probably a newbie question, but I can't seem to find an example of this
anywhere.

If I create a document like so:
curl -XPUT http://localhost:9200/test/testing/1 -d '{
"field1" : "value1"
}'

If I run through the example for creating a percolator like so:

curl -XPUT localhost:9200/_percolator/test/kuku -d '{
"query" : {
"term" : {
"field1" : "value1"
}
}
}'

The documentation says I can percolate a document to see what queries match
by doing this:

curl -XGET localhost:9200/test/type1/_percolate -d '{
"doc" : {
"field1" : "value1"
}
}'

But what if I wanted to percolate the document by passing in it's ID instead
of the entire document? Is this possible? or is it assumed that percolating
is only ever done when a document is created or updated?

Seems like maybe I'm missing something

--

--

3

Hi Radu,

Thanks for the reply. Are you saying that I can pass multiple IDs into the
_percolate XGET and it will attempt to find matches on those? That would
solve my problem.

Just want to clarify because you said "wrap that up in a bool query[1]
along with the other query(es)
that you register as percolator(s)." I actually want to pass IDs of
existing/updated or new documents into ES to see what registered
percolator(s) match.

Cheers

On Tuesday, November 20, 2012 5:16:13 AM UTC-8, Radu Gheorghe wrote:

Hi Allan,

I don't know a direct answer to your question, but a way to achieve
the same thing is to pass the ID(s) you want through an IDs query[0],
and wrap that up in a bool query[1] along with the other query(es)
that you register as percolator(s).

[0] Elasticsearch Platform — Find real-time answers at scale | Elastic
[1] Elasticsearch Platform — Find real-time answers at scale | Elastic

Best regards,
Radu

http://sematext.com/ -- Elasticsearch -- Solr -- Lucene

On Tue, Nov 20, 2012 at 1:08 AM, Allan Mercado <amer...@darelle.com<javascript:>>
wrote:

Hi all,
Probably a newbie question, but I can't seem to find an example of this
anywhere.

If I create a document like so:
curl -XPUT http://localhost:9200/test/testing/1 -d '{
"field1" : "value1"
}'

If I run through the example for creating a percolator like so:

curl -XPUT localhost:9200/_percolator/test/kuku -d '{
"query" : {
"term" : {
"field1" : "value1"
}
}
}'

The documentation says I can percolate a document to see what queries
match
by doing this:

curl -XGET localhost:9200/test/type1/_percolate -d '{
"doc" : {
"field1" : "value1"
}
}'

But what if I wanted to percolate the document by passing in it's ID
instead
of the entire document? Is this possible? or is it assumed that
percolating
is only ever done when a document is created or updated?

Seems like maybe I'm missing something

--

--

4

Hello Allan,

What I meant was that if you have a couple of docs like this:

curl -XPUT localhost:9200/test/test/1 -d '{
"field1":"value1"
}'

curl -XPUT localhost:9200/test/test/2 -d '{
"field1":"value2"
}'

And you'd normally have a percolator like this:

curl -XPUT localhost:9200/_percolator/test/kuku -d '{
"query" : {
"term" : {
"field1" : "value1"
}
}
}'

You can check docs by ID, that also match the query from the
percolator like this:

curl -XPOST localhost:9200/test/_search?pretty=true -d '{
"query": {
"bool": {
"must": [
{
"ids": {
"values": ["1", "2"]
}
},
{
"term" : {
"field1" : "value1"
}
}
]
}
}
}'

What I didn't mention is that obviously this works well when there's
only one "percolator".

If you have many such queries, this road seems a bit ugly to me: you
can put all your percolator-queries in a "should" clause of your bool
query. And then you should get higher score for docs that match more
queries. If you want to know which matched what query, the Explain API
might help:

So there might be more elegant solutions, that I don't see right now.

Best regards,
Radu

http://sematext.com/ -- Elasticsearch -- Solr -- Lucene

On Tue, Nov 20, 2012 at 6:52 PM, Allan Mercado amercado@darelle.com wrote:

Hi Radu,

Thanks for the reply. Are you saying that I can pass multiple IDs into the
_percolate XGET and it will attempt to find matches on those? That would
solve my problem.

Just want to clarify because you said "wrap that up in a bool query[1] along
with the other query(es)
that you register as percolator(s)." I actually want to pass IDs of
existing/updated or new documents into ES to see what registered
percolator(s) match.

Cheers

On Tuesday, November 20, 2012 5:16:13 AM UTC-8, Radu Gheorghe wrote:

Hi Allan,

I don't know a direct answer to your question, but a way to achieve
the same thing is to pass the ID(s) you want through an IDs query[0],
and wrap that up in a bool query[1] along with the other query(es)
that you register as percolator(s).

[0] Elasticsearch Platform — Find real-time answers at scale | Elastic
[1] Elasticsearch Platform — Find real-time answers at scale | Elastic

Best regards,
Radu

http://sematext.com/ -- Elasticsearch -- Solr -- Lucene

On Tue, Nov 20, 2012 at 1:08 AM, Allan Mercado amer...@darelle.com
wrote:

Hi all,
Probably a newbie question, but I can't seem to find an example of this
anywhere.

If I create a document like so:
curl -XPUT http://localhost:9200/test/testing/1 -d '{
"field1" : "value1"
}'

If I run through the example for creating a percolator like so:

curl -XPUT localhost:9200/_percolator/test/kuku -d '{
"query" : {
"term" : {
"field1" : "value1"
}
}
}'

The documentation says I can percolate a document to see what queries
match
by doing this:

curl -XGET localhost:9200/test/type1/_percolate -d '{
"doc" : {
"field1" : "value1"
}
}'

But what if I wanted to percolate the document by passing in it's ID
instead
of the entire document? Is this possible? or is it assumed that
percolating
is only ever done when a document is created or updated?

Seems like maybe I'm missing something

--

--

--

5

Hi Radu,
Thanks again for the response.
Ok, that info makes sense but is not the use case I was hoping for.

Perhaps I can try to explain my use case again. It might be that I'm misunderstanding the purpose of the purcolator functionality.
I would like to be able to:
Register multiple percolator queries, with varying types of search/filter criteria.
Index multiple documents into ES, each with an ID
On creation of a new document, or on update of an existing one, I'd like to be able to test if that document "matches" any of the registered percolator queries. I'd like to do so by providing the document's ID instead of having to pass in the entire document in the body of the request.
so instead of this:
curl -XGET localhost:9200/test/type1/_percolate -d '{"doc" : {"field1" : "value1", 'more fields': 'more values ..... }}'

I'd like to be able to do something like
curl -XGET localhost:9200/test/type1/_percolate -d '{"SOME_PARAM": MY_DOCUMENT_ID}'

Does that make more sense?

Am I understanding what percolator is meant for correctly?

Thanks again.
Cheers

--
Allan Mercado
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)

On Tuesday, November 20, 2012 at 11:56 PM, Radu Gheorghe wrote:

Hello Allan,

What I meant was that if you have a couple of docs like this:

curl -XPUT localhost:9200/test/test/1 -d '{
"field1":"value1"
}'

curl -XPUT localhost:9200/test/test/2 -d '{
"field1":"value2"
}'

And you'd normally have a percolator like this:

curl -XPUT localhost:9200/_percolator/test/kuku -d '{
"query" : {
"term" : {
"field1" : "value1"
}
}
}'

You can check docs by ID, that also match the query from the
percolator like this:

curl -XPOST localhost:9200/test/_search?pretty=true -d '{
"query": {
"bool": {
"must": [
{
"ids": {
"values": ["1", "2"]
}
},
{
"term" : {
"field1" : "value1"
}
}
]
}
}
}'

What I didn't mention is that obviously this works well when there's
only one "percolator".

If you have many such queries, this road seems a bit ugly to me: you
can put all your percolator-queries in a "should" clause of your bool
query. And then you should get higher score for docs that match more
queries. If you want to know which matched what query, the Explain API
might help:
Elasticsearch Platform — Find real-time answers at scale | Elastic

So there might be more elegant solutions, that I don't see right now.

Best regards,
Radu

http://sematext.com/ -- Elasticsearch -- Solr -- Lucene

On Tue, Nov 20, 2012 at 6:52 PM, Allan Mercado <amercado@darelle.com (mailto:amercado@darelle.com)> wrote:

Hi Radu,

Thanks for the reply. Are you saying that I can pass multiple IDs into the
_percolate XGET and it will attempt to find matches on those? That would
solve my problem.

Just want to clarify because you said "wrap that up in a bool query[1] along
with the other query(es)
that you register as percolator(s)." I actually want to pass IDs of
existing/updated or new documents into ES to see what registered
percolator(s) match.

Cheers

On Tuesday, November 20, 2012 5:16:13 AM UTC-8, Radu Gheorghe wrote:

Hi Allan,

I don't know a direct answer to your question, but a way to achieve
the same thing is to pass the ID(s) you want through an IDs query[0],
and wrap that up in a bool query[1] along with the other query(es)
that you register as percolator(s).

[0] Elasticsearch Platform — Find real-time answers at scale | Elastic
[1] Elasticsearch Platform — Find real-time answers at scale | Elastic

Best regards,
Radu

http://sematext.com/ -- Elasticsearch -- Solr -- Lucene

On Tue, Nov 20, 2012 at 1:08 AM, Allan Mercado <amer...@darelle.com (http://darelle.com)>
wrote:

Hi all,
Probably a newbie question, but I can't seem to find an example of this
anywhere.

If I create a document like so:
curl -XPUT http://localhost:9200/test/testing/1 -d '{
"field1" : "value1"
}'

If I run through the example for creating a percolator like so:

curl -XPUT localhost:9200/_percolator/test/kuku -d '{
"query" : {
"term" : {
"field1" : "value1"
}
}
}'

The documentation says I can percolate a document to see what queries
match
by doing this:

curl -XGET localhost:9200/test/type1/_percolate -d '{
"doc" : {
"field1" : "value1"
}
}'

But what if I wanted to percolate the document by passing in it's ID
instead
of the entire document? Is this possible? or is it assumed that
percolating
is only ever done when a document is created or updated?

Seems like maybe I'm missing something

--

--

--

--

6

Hi Allan,

On Wed, Nov 21, 2012 at 10:32 AM, Allan Mercado amercado@darelle.com wrote:

Hi Radu,
Thanks again for the response.
Ok, that info makes sense but is not the use case I was hoping for.

Perhaps I can try to explain my use case again. It might be that I'm
misunderstanding the purpose of the purcolator functionality.
I would like to be able to:

Register multiple percolator queries, with varying types of search/filter
criteria.
Index multiple documents into ES, each with an ID
On creation of a new document, or on update of an existing one, I'd like to
be able to test if that document "matches" any of the registered percolator
queries.

Well, you can do that while indexing. Take a look here for an example:

So you define a percolator:
$ curl -XPUT localhost:9200/_percolator/twitter/elasticsearch -d '{
"query" : {
"field" : {
"message" : "elasticsearch"
}
}
}'

And then when you index, you specify the percolator (wildcards also work):
$ curl -XPUT 'localhost:9200/twitter/tweet/1?percolate=*' -d '{
"message" : "this new elasticsearch percolator feature is nice, borat style"
}'

So you'll get your document indexed and your percolator matching. And
it should also work with bulks.

I'd like to do so by providing the document's ID instead of having
to pass in the entire document in the body of the request.

Yeah, I understood that it's what you wanted, but I'm not aware of a
way to achieve exactly that. So I was suggesting other ways.

so instead of this:

curl -XGET localhost:9200/test/type1/_percolate -d '{"doc" : {"field1" :
"value1", 'more fields': 'more values ..... }}'

I'd like to be able to do something like

curl -XGET localhost:9200/test/type1/_percolate -d '{"SOME_PARAM":
MY_DOCUMENT_ID}'

Does that make more sense?

Yes, it made sense from your first post.

Am I understanding what percolator is meant for correctly?

I think so. Percolators are just queries you register to ES and you
can send documents to them and see which ones match.

Best regards,
Radu

http://sematext.com/ -- Elasticsearch -- Solr -- Lucene

Thanks again.
Cheers

--
Allan Mercado
Sent with Sparrow

On Tuesday, November 20, 2012 at 11:56 PM, Radu Gheorghe wrote:

Hello Allan,

What I meant was that if you have a couple of docs like this:

curl -XPUT localhost:9200/test/test/1 -d '{
"field1":"value1"
}'

curl -XPUT localhost:9200/test/test/2 -d '{
"field1":"value2"
}'

And you'd normally have a percolator like this:

curl -XPUT localhost:9200/_percolator/test/kuku -d '{
"query" : {
"term" : {
"field1" : "value1"
}
}
}'

You can check docs by ID, that also match the query from the
percolator like this:

curl -XPOST localhost:9200/test/_search?pretty=true -d '{
"query": {
"bool": {
"must": [
{
"ids": {
"values": ["1", "2"]
}
},
{
"term" : {
"field1" : "value1"
}
}
]
}
}
}'

What I didn't mention is that obviously this works well when there's
only one "percolator".

If you have many such queries, this road seems a bit ugly to me: you
can put all your percolator-queries in a "should" clause of your bool
query. And then you should get higher score for docs that match more
queries. If you want to know which matched what query, the Explain API
might help:
Elasticsearch Platform — Find real-time answers at scale | Elastic

So there might be more elegant solutions, that I don't see right now.

Best regards,
Radu

http://sematext.com/ -- Elasticsearch -- Solr -- Lucene

On Tue, Nov 20, 2012 at 6:52 PM, Allan Mercado amercado@darelle.com wrote:

Hi Radu,

Thanks for the reply. Are you saying that I can pass multiple IDs into the
_percolate XGET and it will attempt to find matches on those? That would
solve my problem.

Just want to clarify because you said "wrap that up in a bool query[1] along
with the other query(es)
that you register as percolator(s)." I actually want to pass IDs of
existing/updated or new documents into ES to see what registered
percolator(s) match.

Cheers

On Tuesday, November 20, 2012 5:16:13 AM UTC-8, Radu Gheorghe wrote:

Hi Allan,

I don't know a direct answer to your question, but a way to achieve
the same thing is to pass the ID(s) you want through an IDs query[0],
and wrap that up in a bool query[1] along with the other query(es)
that you register as percolator(s).

[0] Elasticsearch Platform — Find real-time answers at scale | Elastic
[1] Elasticsearch Platform — Find real-time answers at scale | Elastic

Best regards,
Radu

http://sematext.com/ -- Elasticsearch -- Solr -- Lucene

On Tue, Nov 20, 2012 at 1:08 AM, Allan Mercado amer...@darelle.com
wrote:

Hi all,
Probably a newbie question, but I can't seem to find an example of this
anywhere.

If I create a document like so:
curl -XPUT http://localhost:9200/test/testing/1 -d '{
"field1" : "value1"
}'

If I run through the example for creating a percolator like so:

curl -XPUT localhost:9200/_percolator/test/kuku -d '{
"query" : {
"term" : {
"field1" : "value1"
}
}
}'

The documentation says I can percolate a document to see what queries
match
by doing this:

curl -XGET localhost:9200/test/type1/_percolate -d '{
"doc" : {
"field1" : "value1"
}
}'

But what if I wanted to percolate the document by passing in it's ID
instead
of the entire document? Is this possible? or is it assumed that
percolating
is only ever done when a document is created or updated?

Seems like maybe I'm missing something

--

--

--

--

--

7

Hi Radu,

Ok, great. Thanks for the clarification. I just wanted to make sure I was understanding correctly :slight_smile:
I really appreciate your help!

Cheers
Allan

On 2012-11-21, at 5:11 AM, Radu Gheorghe wrote:

Hi Allan,

On Wed, Nov 21, 2012 at 10:32 AM, Allan Mercado amercado@darelle.com wrote:

Hi Radu,
Thanks again for the response.
Ok, that info makes sense but is not the use case I was hoping for.

Perhaps I can try to explain my use case again. It might be that I'm
misunderstanding the purpose of the purcolator functionality.
I would like to be able to:

Register multiple percolator queries, with varying types of search/filter
criteria.
Index multiple documents into ES, each with an ID
On creation of a new document, or on update of an existing one, I'd like to
be able to test if that document "matches" any of the registered percolator
queries.

Well, you can do that while indexing. Take a look here for an example:
Elasticsearch Platform — Find real-time answers at scale | Elastic

So you define a percolator:
$ curl -XPUT localhost:9200/_percolator/twitter/elasticsearch -d '{
"query" : {
"field" : {
"message" : "elasticsearch"
}
}
}'

And then when you index, you specify the percolator (wildcards also work):
$ curl -XPUT 'localhost:9200/twitter/tweet/1?percolate=*' -d '{
"message" : "this new elasticsearch percolator feature is nice, borat style"
}'

So you'll get your document indexed and your percolator matching. And
it should also work with bulks.

I'd like to do so by providing the document's ID instead of having
to pass in the entire document in the body of the request.

Yeah, I understood that it's what you wanted, but I'm not aware of a
way to achieve exactly that. So I was suggesting other ways.

so instead of this:

curl -XGET localhost:9200/test/type1/_percolate -d '{"doc" : {"field1" :
"value1", 'more fields': 'more values ..... }}'

I'd like to be able to do something like

curl -XGET localhost:9200/test/type1/_percolate -d '{"SOME_PARAM":
MY_DOCUMENT_ID}'

Does that make more sense?

Yes, it made sense from your first post.

Am I understanding what percolator is meant for correctly?

I think so. Percolators are just queries you register to ES and you
can send documents to them and see which ones match.

Best regards,
Radu

http://sematext.com/ -- Elasticsearch -- Solr -- Lucene

Thanks again.
Cheers

--
Allan Mercado
Sent with Sparrow

On Tuesday, November 20, 2012 at 11:56 PM, Radu Gheorghe wrote:

Hello Allan,

What I meant was that if you have a couple of docs like this:

curl -XPUT localhost:9200/test/test/1 -d '{
"field1":"value1"
}'

curl -XPUT localhost:9200/test/test/2 -d '{
"field1":"value2"
}'

And you'd normally have a percolator like this:

curl -XPUT localhost:9200/_percolator/test/kuku -d '{
"query" : {
"term" : {
"field1" : "value1"
}
}
}'

You can check docs by ID, that also match the query from the
percolator like this:

curl -XPOST localhost:9200/test/_search?pretty=true -d '{
"query": {
"bool": {
"must": [
{
"ids": {
"values": ["1", "2"]
}
},
{
"term" : {
"field1" : "value1"
}
}
]
}
}
}'

What I didn't mention is that obviously this works well when there's
only one "percolator".

If you have many such queries, this road seems a bit ugly to me: you
can put all your percolator-queries in a "should" clause of your bool
query. And then you should get higher score for docs that match more
queries. If you want to know which matched what query, the Explain API
might help:
Elasticsearch Platform — Find real-time answers at scale | Elastic

So there might be more elegant solutions, that I don't see right now.

Best regards,
Radu

http://sematext.com/ -- Elasticsearch -- Solr -- Lucene

On Tue, Nov 20, 2012 at 6:52 PM, Allan Mercado amercado@darelle.com wrote:

Hi Radu,

Thanks for the reply. Are you saying that I can pass multiple IDs into the
_percolate XGET and it will attempt to find matches on those? That would
solve my problem.

Just want to clarify because you said "wrap that up in a bool query[1] along
with the other query(es)
that you register as percolator(s)." I actually want to pass IDs of
existing/updated or new documents into ES to see what registered
percolator(s) match.

Cheers

On Tuesday, November 20, 2012 5:16:13 AM UTC-8, Radu Gheorghe wrote:

Hi Allan,

I don't know a direct answer to your question, but a way to achieve
the same thing is to pass the ID(s) you want through an IDs query[0],
and wrap that up in a bool query[1] along with the other query(es)
that you register as percolator(s).

[0] Elasticsearch Platform — Find real-time answers at scale | Elastic
[1] Elasticsearch Platform — Find real-time answers at scale | Elastic

Best regards,
Radu

http://sematext.com/ -- Elasticsearch -- Solr -- Lucene

On Tue, Nov 20, 2012 at 1:08 AM, Allan Mercado amer...@darelle.com
wrote:

Hi all,
Probably a newbie question, but I can't seem to find an example of this
anywhere.

If I create a document like so:
curl -XPUT http://localhost:9200/test/testing/1 -d '{
"field1" : "value1"
}'

If I run through the example for creating a percolator like so:

curl -XPUT localhost:9200/_percolator/test/kuku -d '{
"query" : {
"term" : {
"field1" : "value1"
}
}
}'

The documentation says I can percolate a document to see what queries
match
by doing this:

curl -XGET localhost:9200/test/type1/_percolate -d '{
"doc" : {
"field1" : "value1"
}
}'

But what if I wanted to percolate the document by passing in it's ID
instead
of the entire document? Is this possible? or is it assumed that
percolating
is only ever done when a document is created or updated?

Seems like maybe I'm missing something

--

--

--

--

--

--