Percolator with lookup terms filter not working?

Alexander_Jiteg · October 30, 2014, 8:59am

Hi!

I'm trying to use a lookup terms filter for percolation but for some reason
I'm not getting any matches when percolating documents that should match
the registered percolator.

Example:

gist.github.com

https://gist.github.com/alexndr79/760314b8b5f49157a839#file-percolation_with_terms_lookup-txt

percolation_with_terms_lookup.txt

POST /filter/filter/1
{"str":["1","2","3"]}

POST /test/.percolator/1
{"query":{"filtered":{"filter":{"and":{"filters":[{"terms":{"myfield":{"index":"filter","type":"filter","id":"1","path":"str", "cache":"false"}}}]}}}}}

POST /test/test/_percolate
{"doc": {"myfield": "1"}}

I have noted that if I try to index the same percolator a second time after
the first percolation (that gives not matches) it seems that following
percolations will give the expected result.

I'm running ES 1.3.4.

Suggestions?

/Alex

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/2e2ebd43-0afd-4330-b4f2-5ed2c00097c1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Alexander_Jiteg · October 30, 2014, 12:23pm

Seems that If I index a document with the given type first, it works. That
is probably why my second run works but not the first.

On Thursday, October 30, 2014 9:59:49 AM UTC+1, Alexander Jiteg wrote:

Hi!

I'm trying to use a lookup terms filter for percolation but for some
reason I'm not getting any matches when percolating documents that should
match the registered percolator.

Example:

Percolation with terms look up · GitHub

I have noted that if I try to index the same percolator a second time
after the first percolation (that gives not matches) it seems that
following percolations will give the expected result.

I'm running ES 1.3.4.

Suggestions?

/Alex

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b185c84f-328f-4f4b-85c6-50a746919cb6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

mvg · November 3, 2014, 9:20am

In the case of percolator queries are indexed before actual data is
evaluated, which triggers the automatic addition of fields in mappings.
Most of the times this is ok, but there are a number of queries that
require the field mapping to exist before the percolator query is indexed.

Because of this from 1.4.0 and onwards it is required that a field mapping
exists for fields used in percolator queries:

I hope this explains.

On 30 October 2014 13:23, Alexander Jiteg alexander.jiteg@tickstar.com
wrote:

Seems that If I index a document with the given type first, it works.
That is probably why my second run works but not the first.

On Thursday, October 30, 2014 9:59:49 AM UTC+1, Alexander Jiteg wrote:

Hi!

I'm trying to use a lookup terms filter for percolation but for some
reason I'm not getting any matches when percolating documents that should
match the registered percolator.

Example:
Percolation with terms look up · GitHub
percolation_with_terms_lookup-txt

I have noted that if I try to index the same percolator a second time
after the first percolation (that gives not matches) it seems that
following percolations will give the expected result.

I'm running ES 1.3.4.

Suggestions?

/Alex

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/b185c84f-328f-4f4b-85c6-50a746919cb6%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/b185c84f-328f-4f4b-85c6-50a746919cb6%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Met vriendelijke groet,

Martijn van Groningen

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CA%2BA76Txpxivfh1bn4K6dqcHvz5qxGd_0xPOPgvDbxn-p%3Dgx%3DYA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Alexander_Jiteg · November 3, 2014, 9:38am

Ok. Thanks for the clarification.

On Monday, November 3, 2014 10:20:38 AM UTC+1, Martijn v Groningen wrote:

In the case of percolator queries are indexed before actual data is
evaluated, which triggers the automatic addition of fields in mappings.
Most of the times this is ok, but there are a number of queries that
require the field mapping to exist before the percolator query is indexed.

Because of this from 1.4.0 and onwards it is required that a field mapping
exists for fields used in percolator queries:
Add a `index.query.parse.allow_unmapped_fields` setting that fails if queries refer to unmapped fields. by martijnvg · Pull Request #6928 · elastic/elasticsearch · GitHub

I hope this explains.

On 30 October 2014 13:23, Alexander Jiteg <alexand...@tickstar.com
<javascript:>> wrote:

Seems that If I index a document with the given type first, it works.
That is probably why my second run works but not the first.

On Thursday, October 30, 2014 9:59:49 AM UTC+1, Alexander Jiteg wrote:

Hi!

I'm trying to use a lookup terms filter for percolation but for some
reason I'm not getting any matches when percolating documents that should
match the registered percolator.

Example:
Percolation with terms look up · GitHub
percolation_with_terms_lookup-txt

I have noted that if I try to index the same percolator a second time
after the first percolation (that gives not matches) it seems that
following percolations will give the expected result.

I'm running ES 1.3.4.

Suggestions?

/Alex

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/b185c84f-328f-4f4b-85c6-50a746919cb6%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/b185c84f-328f-4f4b-85c6-50a746919cb6%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Met vriendelijke groet,

Martijn van Groningen

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9d195f44-21b0-4376-a171-e2dc7fbce7e8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Alexander_Jiteg · November 13, 2014, 3:44pm

Follow up question; What If I change the terms in the looked up filter
after the percolator has been created? Seems that the terms are cached but
even though I have experimented with cache invalidation the only thing I
got working so far is deleting/recreating the percolator for the new terms
to kick in.

On Monday, November 3, 2014 10:20:38 AM UTC+1, Martijn v Groningen wrote:

In the case of percolator queries are indexed before actual data is
evaluated, which triggers the automatic addition of fields in mappings.
Most of the times this is ok, but there are a number of queries that
require the field mapping to exist before the percolator query is indexed.

Because of this from 1.4.0 and onwards it is required that a field mapping
exists for fields used in percolator queries:
Add a `index.query.parse.allow_unmapped_fields` setting that fails if queries refer to unmapped fields. by martijnvg · Pull Request #6928 · elastic/elasticsearch · GitHub

I hope this explains.

On 30 October 2014 13:23, Alexander Jiteg <alexand...@tickstar.com
<javascript:>> wrote:

Seems that If I index a document with the given type first, it works.
That is probably why my second run works but not the first.

On Thursday, October 30, 2014 9:59:49 AM UTC+1, Alexander Jiteg wrote:

Hi!

I'm trying to use a lookup terms filter for percolation but for some
reason I'm not getting any matches when percolating documents that should
match the registered percolator.

Example:
Percolation with terms look up · GitHub
percolation_with_terms_lookup-txt

I have noted that if I try to index the same percolator a second time
after the first percolation (that gives not matches) it seems that
following percolations will give the expected result.

I'm running ES 1.3.4.

Suggestions?

/Alex

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/b185c84f-328f-4f4b-85c6-50a746919cb6%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/b185c84f-328f-4f4b-85c6-50a746919cb6%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Met vriendelijke groet,

Martijn van Groningen

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e625515b-f434-4db4-b928-211f620db03a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Alexander_Jiteg · November 25, 2014, 2:09pm

No suggestions?

/A

On Thursday, November 13, 2014 4:44:35 PM UTC+1, Alexander Jiteg wrote:

Follow up question; What If I change the terms in the looked up filter
after the percolator has been created? Seems that the terms are cached but
even though I have experimented with cache invalidation the only thing I
got working so far is deleting/recreating the percolator for the new terms
to kick in.

On Monday, November 3, 2014 10:20:38 AM UTC+1, Martijn v Groningen wrote:

In the case of percolator queries are indexed before actual data is
evaluated, which triggers the automatic addition of fields in mappings.
Most of the times this is ok, but there are a number of queries that
require the field mapping to exist before the percolator query is indexed.

Because of this from 1.4.0 and onwards it is required that a field
mapping exists for fields used in percolator queries:
Add a `index.query.parse.allow_unmapped_fields` setting that fails if queries refer to unmapped fields. by martijnvg · Pull Request #6928 · elastic/elasticsearch · GitHub

I hope this explains.

On 30 October 2014 13:23, Alexander Jiteg alexand...@tickstar.com
wrote:

Seems that If I index a document with the given type first, it works.
That is probably why my second run works but not the first.

On Thursday, October 30, 2014 9:59:49 AM UTC+1, Alexander Jiteg wrote:

Hi!

I'm trying to use a lookup terms filter for percolation but for some
reason I'm not getting any matches when percolating documents that should
match the registered percolator.

Example:
Percolation with terms look up · GitHub
percolation_with_terms_lookup-txt

I have noted that if I try to index the same percolator a second time
after the first percolation (that gives not matches) it seems that
following percolations will give the expected result.

I'm running ES 1.3.4.

Suggestions?

/Alex

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/b185c84f-328f-4f4b-85c6-50a746919cb6%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/b185c84f-328f-4f4b-85c6-50a746919cb6%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Met vriendelijke groet,

Martijn van Groningen

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/56b2d3ba-757a-46f7-9db1-70f97acc5e1c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.