Percolate existing document using query from another index


(Poomrat Boonyawong) #1

Hi, I am trying to follow advice
in http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-percolate.html#_dedicated_percolator_index
but percolating on existing document.
This for some reason doesn't work for me in ES 1.1.0.

The same percolator works (match document) if it is in the same index as
target document.

Here's my test document

curl -XGET

http://pvcm-emsdata01.amers1b.ciscloud:9200/logstash-2014.04.18/apache_access/HAtwipHtSnOmr8ZUn4Q4tw

{

"_index" : "logstash-2014.04.18",

"_type" : "apache_access",

"_id" : "HAtwipHtSnOmr8ZUn4Q4tw",

"_version" : 1,

"found" : true,

"_source" : {

"@version" : "1",

"@timestamp" : "2014-04-18T08:50:48.461+00:00",

"type" : "apache_access",

"tags" : ["input_nxlog_json", "logstash-ingestion-node-PVCM-EMSINGS01",

"logstash-index-node-PVCM-EMSINDX01", "_grokparsefailure"],

"host" : "pvcm-emshead01",

"message" : "10.42.78.81 - - [18/Apr/2014:08:50:47 +0000] GET

/monitor/events/css/bootstrap-responsive.min.css HTTP/1.1 304 -",

"itic-lsg-code" : "HEAD",

...

}

}

If I add percolator like this

curl -XPUT

http://pvcm-emsdata01.amers1b.ciscloud:9200/logstash-2014.04.18/.percolator/head_lsg_in_logstash_index
-d '{ "query" : { "bool" : { "must" : [{ "match" : { "host" :
"pvcm-emshead01" } }, { "match" : { "itic-lsg-code" : "HEAD" } } ] } } }'

It match this document fine

curl -XGET
http://pvcm-emsdata01.amers1b.ciscloud:9200/logstash-2014.04.18/apache_access/HAtwipHtSnOmr8ZUn4Q4tw/_percolate

{

"took" : 2,

"_shards" : {

"total" : 2,

"successful" : 2,

"failed" : 0

},

"total" : 1,

"matches" : [{

"_index" : "logstash-2014.04.18",

"_id" : "head_lsg_in_logstash_index"

}

]

}

But if I add the same percolator to dedicated index

curl -XPUT

http://pvcm-emsdata01.amers1b.ciscloud:9200/percolator/.percolator/head_lsg
-d '{

"query" : {

"bool" : {

"must" : [{

"match" : {

"host" : "pvcm-emshead01"

}

}, {

"match" : {

"itic-lsg-code" : "HEAD"

}

}

]

}

}

}'

I can't find any combination that use percolator in dedicated index against
existing document. Here's combination I've tried to no avail

  • Set percolate_index and percolate_type and call percolate against
    existing document - get match from percolator in document's index instead

curl -XGET

http://pvcm-emsdata01.amers1b.ciscloud:9200/logstash-2014.04.18/apache_access/HAtwipHtSnOmr8ZUn4Q4tw/_percolate
-d'{

"percolate_index" : "percolator",

"percolate_type" : "apache_access"

}'

{"took":2,"_shards":{"total":2,"successful":2,"failed":0},"total":1,"matches":[{"_index":"logstash-2014.04.18","_id":"head_lsg_in_logstash_index"}]}

  • Try calling percolate from dedicated index - error at run time

curl -XPOST

http://pvcm-emsdata01.amers1b.ciscloud:9200/percolator/.percolator/_percolate
-d'{

"id": "HAtwipHtSnOmr8ZUn4Q4tw",

"percolate_index" : "logstash-2014.04.18",

"percolate_type" : "apache_access"

}'

{"took":1,"_shards":{"total":2,"successful":0,"failed":2,"failures":[{"index":"percolator","shard":0,"reason":"BroadcastShardOperationFailedException[[percolator][0]

]; nested: PercolateException[failed to percolate]; nested:
ElasticsearchIllegalArgumentException[Nothing to percolate];
"},{"index":"percolator","shard":1,"reason":"BroadcastShardOperationFailedException[[percolator][1]
]; nested: PercolateException[failed to percolate]; nested:
ElasticsearchIllegalArgumentException[Nothing to percolate];
"}]},"total":0,"matches":[]}

Although if I try that percolator against new document (using "doc"), it
matches just fine.

curl -XGET

http://pvcm-emsdata01.amers1b.ciscloud:9200/percolator/apache_access/_percolate
-d'{

"doc" : {

"type": "apache_access",

  "tags":       [

     "input_nxlog_json",

     "logstash-ingestion-node-PVCM-EMSINGS01",

     "logstash-index-node-PVCM-EMSINDX01",

     "_grokparsefailure"

  ],

  "host": "pvcm-emshead01",

  "message": "10.42.78.81 - - [21/Apr/2014:09:25:47 +0000] GET 

/monitor/events/css/bootstrap.light.min.css HTTP/1.1 304 -",

  "itic-lsg-code": "HEAD"

}

}'

{"took":2,"_shards":{"total":2,"successful":2,"failed":0},"total":2,"matches":[{"_index":"percolator","_id":"head_lsg"},{"_index":"percolator","_id":"apache_access_get_default"}]}

Can someone show me how this use case can be done or is it even supported?

Many thanks in advance

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/3eabdc40-9eb2-4f57-9f01-30c94fe9a3e7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Binh Ly-2) #2

If you have a doc in an index A and you want to percolate it against
queries in index B, you'll need to specify the percolate_index (and
potentially percolate_type) url param:

curl -XPOST
"http://localhost:9200/a/doc/1/_percolate?pretty&percolate_index=b"

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5526868d-5b1f-46a2-b488-a22357f3ed10%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Poomrat Boonyawong) #3

Now that is just what I need.
Thanks a ton for this pointer.

On Tuesday, April 22, 2014 5:06:00 AM UTC+7, Binh Ly wrote:

If you have a doc in an index A and you want to percolate it against
queries in index B, you'll need to specify the percolate_index (and
potentially percolate_type) url param:

curl -XPOST "
http://localhost:9200/a/doc/1/_percolate?pretty&percolate_index=b"

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/663672a2-8bf0-4fe6-a73b-740004c3ccb3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(system) #4