(mvel) filtering returned facets with 'script'

Tomasz_Kloc · January 7, 2012, 10:19am

hi,

I have a filtered query with facet. I want to get facets only for
explicit terms and avoid using size: X or all_terms (and filtering
facets on application side).

i've succeed to do that with 'script':

{'facets': {'category_ids': {'terms': {'field': 'category_ids',
'script': 'term == 9560'}}}}

the only thing i can't figure out is how to write mvel's script which
checks if term is in some list/array. Python example:

term in [1,2,3]

i've also tried to use python syntax adding 'lang' parameter with values
'python'/'lang-python', but query failed with exception:
ElasticSearchIllegalArgumentException[script_lang not supported
[lang-python]]
or
ElasticSearchIllegalArgumentException[script_lang not supported [python]]

any suggestions?

Tomasz_Kloc · January 7, 2012, 10:37am

Nevermind. I've succeed with: "script": "[9560, 9561].contains(term)"

hi,

I have a filtered query with facet. I want to get facets only for
explicit terms and avoid using size: X or all_terms (and filtering
facets on application side).

i've succeed to do that with 'script':

{'facets': {'category_ids': {'terms': {'field': 'category_ids',
'script': 'term == 9560'}}}}

the only thing i can't figure out is how to write mvel's script which
checks if term is in some list/array. Python example:

term in [1,2,3]

i've also tried to use python syntax adding 'lang' parameter with
values 'python'/'lang-python', but query failed with exception:
ElasticSearchIllegalArgumentException[script_lang not supported
[lang-python]]
or
ElasticSearchIllegalArgumentException[script_lang not supported [python]]

any suggestions?

kimchy · January 7, 2012, 7:43pm

I suggest passing the list of terms as a parameter to the script, so it
won't be compiled each time.

On Sat, Jan 7, 2012 at 12:37 PM, Tomasz Kloc tomek.kloc.iit@gmail.comwrote:

Nevermind. I've succeed with: "script": "[9560, 9561].contains(term)"

hi,

I have a filtered query with facet. I want to get facets only for
explicit terms and avoid using size: X or all_terms (and filtering facets
on application side).

i've succeed to do that with 'script':

{'facets': {'category_ids': {'terms': {'field': 'category_ids',
'script': 'term == 9560'}}}}

the only thing i can't figure out is how to write mvel's script which
checks if term is in some list/array. Python example:

term in [1,2,3]

i've also tried to use python syntax adding 'lang' parameter with values
'python'/'lang-python', but query failed with exception:
ElasticSearchIllegalArgumentException[script_lang not supported
[lang-python]]
or
ElasticSearchIllegalArgumentException[script_lang not supported
[python]]

any suggestions?

Tomasz_Kloc · January 8, 2012, 9:51am

Unfortunatelly, terms are different on each query.
Also, my solution doesn't work, because for some term's list i get
exception:

nested: ClassCastException[org.elasticsearch.common.mvel2.util.FastList
cannot be cast to java.util.ArrayList

This one will throw exception:
"script": "term in ['3669', '3670', '3676', '3677', '3688', '3712',
'3739', '3748', '3749']"

but this one not:
"script": "term in ['3669', '3670', '3676', '3677', '3688', '3712',
'3739', '3748']"

I had no idea what was going on (array length is not an issue), so I've
changed script to:

"script": "term == 5823 || term == 5824 || ..."

On 07.01.2012 20:43, Shay Banon wrote:

I suggest passing the list of terms as a parameter to the script, so
it won't be compiled each time.

On Sat, Jan 7, 2012 at 12:37 PM, Tomasz Kloc <tomek.kloc.iit@gmail.com
mailto:tomek.kloc.iit@gmail.com> wrote:

Nevermind. I've succeed with: "script": "[9560, 9561].contains(term)"


    hi,

    I have a filtered query with facet. I  want to get facets only
    for explicit terms and  avoid using size: X or all_terms (and
    filtering facets on application side).

    i've succeed to do that with 'script':

    {'facets': {'category_ids': {'terms': {'field': 'category_ids',
       'script': 'term == 9560'}}}}

    the only thing i can't figure out is how to write mvel's
    script which checks if *term* is in some list/array. Python
    example:

    term in [1,2,3]

    i've also tried to use python syntax adding 'lang' parameter
    with values 'python'/'lang-python', but query failed with
    exception:
    ElasticSearchIllegalArgumentException[script_lang not
    supported [lang-python]]
    or
    ElasticSearchIllegalArgumentException[script_lang not
    supported [python]]


    any suggestions?

kimchy · January 8, 2012, 10:11am

Thats what the params are for, pass a param to the script called terms,
which is a json map, with a key/field as the term, and value is anything
(true is best). Then, can use the terms parameter in the script to do
containsKey (since it a map) and it will be a hash/map based lookup instead
of searching through a list.

On Sun, Jan 8, 2012 at 11:51 AM, Tomasz Kloc tomek.kloc.iit@gmail.comwrote:

**
Unfortunatelly, terms are different on each query.
Also, my solution doesn't work, because for some term's list i get
exception:

nested: ClassCastException[org.elasticsearch.common.mvel2.util.FastList
cannot be cast to java.util.ArrayList

This one will throw exception:
"script": "term in ['3669', '3670', '3676', '3677', '3688', '3712',
'3739', '3748', '3749']"

but this one not:
"script": "term in ['3669', '3670', '3676', '3677', '3688', '3712',
'3739', '3748']"

I had no idea what was going on (array length is not an issue), so I've
changed script to:

"script": "term == 5823 || term == 5824 || ..."

On 07.01.2012 20:43, Shay Banon wrote:

I suggest passing the list of terms as a parameter to the script, so it
won't be compiled each time.

On Sat, Jan 7, 2012 at 12:37 PM, Tomasz Kloc tomek.kloc.iit@gmail.comwrote:

Nevermind. I've succeed with: "script": "[9560, 9561].contains(term)"

hi,

I have a filtered query with facet. I want to get facets only for
explicit terms and avoid using size: X or all_terms (and filtering facets
on application side).

i've succeed to do that with 'script':

{'facets': {'category_ids': {'terms': {'field': 'category_ids',
'script': 'term == 9560'}}}}

the only thing i can't figure out is how to write mvel's script which
checks if term is in some list/array. Python example:

term in [1,2,3]

i've also tried to use python syntax adding 'lang' parameter with values
'python'/'lang-python', but query failed with exception:
ElasticSearchIllegalArgumentException[script_lang not supported
[lang-python]]
or
ElasticSearchIllegalArgumentException[script_lang not supported [python]]

any suggestions?

Nate_Bauernfeind · November 8, 2013, 9:06pm

This suggestion is gold. We ran into this situation where it would only
occur on certain shards resulting in a portion of all results (which was
hard for us to detect originally; now we'll be looking for similar kinds of
issues).

Though it is frustrating to see the "nested:
ClassCastException[org.elasticsearch.common.mvel2.util.FastList cannot be
cast to java.util.ArrayList" exception.

Does anyone know what the underlying reason is for this and how one might
avoid such a situation?

There's another exception that we've seen where a "StringTermsFacet" can't
be merged with an "IntTermsFacet" (or any numerical facet; due to a class
cast exception) when, in our case, the Strings facets are always empty
(dynamic fields of numerical types were being faceted on). We've gone
through great lengths to make static mappings of all dynamic fields that we
might sort or facet on.

And it always feel kind of silly that these things occur only sometimes.
Certain conditions must be met (query across indexes) and there are always
safe and sane choices that ES could do on our behalf, but doesn't.

On Sunday, January 8, 2012 4:11:52 AM UTC-6, kimchy wrote:

Thats what the params are for, pass a param to the script called terms,
which is a json map, with a key/field as the term, and value is anything
(true is best). Then, can use the terms parameter in the script to do
containsKey (since it a map) and it will be a hash/map based lookup instead
of searching through a list.

On Sun, Jan 8, 2012 at 11:51 AM, Tomasz Kloc <tomek.k...@gmail.com<javascript:>

wrote:

Unfortunatelly, terms are different on each query.
Also, my solution doesn't work, because for some term's list i get
exception:

nested: ClassCastException[org.elasticsearch.common.mvel2.util.FastList
cannot be cast to java.util.ArrayList

This one will throw exception:
"script": "term in ['3669', '3670', '3676', '3677', '3688', '3712',
'3739', '3748', '3749']"

but this one not:
"script": "term in ['3669', '3670', '3676', '3677', '3688', '3712',
'3739', '3748']"

I had no idea what was going on (array length is not an issue), so I've
changed script to:

"script": "term == 5823 || term == 5824 || ..."

On 07.01.2012 20:43, Shay Banon wrote:

I suggest passing the list of terms as a parameter to the script, so it
won't be compiled each time.

On Sat, Jan 7, 2012 at 12:37 PM, Tomasz Kloc <tomek.k...@gmail.com<javascript:>

wrote:

Nevermind. I've succeed with: "script": "[9560, 9561].contains(term)"

hi,

I have a filtered query with facet. I want to get facets only for
explicit terms and avoid using size: X or all_terms (and filtering facets
on application side).

i've succeed to do that with 'script':

{'facets': {'category_ids': {'terms': {'field': 'category_ids',
'script': 'term == 9560'}}}}

the only thing i can't figure out is how to write mvel's script which
checks if term is in some list/array. Python example:

term in [1,2,3]

i've also tried to use python syntax adding 'lang' parameter with
values 'python'/'lang-python', but query failed with exception:
ElasticSearchIllegalArgumentException[script_lang not supported
[lang-python]]
or
ElasticSearchIllegalArgumentException[script_lang not supported
[python]]

any suggestions?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Clinton_Gormley · November 9, 2013, 11:10am

Can you provide a recreation and a stacktrace (from the logs? It'd be good
to figure out what is causing this error.

On Friday, 8 November 2013 22:06:33 UTC+1, Nate Bauernfeind wrote:

This suggestion is gold. We ran into this situation where it would only
occur on certain shards resulting in a portion of all results (which was
hard for us to detect originally; now we'll be looking for similar kinds of
issues).

Though it is frustrating to see the "nested:
ClassCastException[org.elasticsearch.common.mvel2.util.FastList cannot be
cast to java.util.ArrayList" exception.

Does anyone know what the underlying reason is for this and how one might
avoid such a situation?

There's another exception that we've seen where a "StringTermsFacet" can't
be merged with an "IntTermsFacet" (or any numerical facet; due to a class
cast exception) when, in our case, the Strings facets are always empty
(dynamic fields of numerical types were being faceted on). We've gone
through great lengths to make static mappings of all dynamic fields that we
might sort or facet on.

And it always feel kind of silly that these things occur only sometimes.
Certain conditions must be met (query across indexes) and there are always
safe and sane choices that ES could do on our behalf, but doesn't.

On Sunday, January 8, 2012 4:11:52 AM UTC-6, kimchy wrote:

Thats what the params are for, pass a param to the script called terms,
which is a json map, with a key/field as the term, and value is anything
(true is best). Then, can use the terms parameter in the script to do
containsKey (since it a map) and it will be a hash/map based lookup instead
of searching through a list.

On Sun, Jan 8, 2012 at 11:51 AM, Tomasz Kloc tomek.k...@gmail.comwrote:

Unfortunatelly, terms are different on each query.
Also, my solution doesn't work, because for some term's list i get
exception:

nested: ClassCastException[org.elasticsearch.common.mvel2.util.FastList
cannot be cast to java.util.ArrayList

This one will throw exception:
"script": "term in ['3669', '3670', '3676', '3677', '3688', '3712',
'3739', '3748', '3749']"

but this one not:
"script": "term in ['3669', '3670', '3676', '3677', '3688', '3712',
'3739', '3748']"

I had no idea what was going on (array length is not an issue), so I've
changed script to:

"script": "term == 5823 || term == 5824 || ..."

On 07.01.2012 20:43, Shay Banon wrote:

I suggest passing the list of terms as a parameter to the script, so it
won't be compiled each time.

On Sat, Jan 7, 2012 at 12:37 PM, Tomasz Kloc tomek.k...@gmail.comwrote:

Nevermind. I've succeed with: "script": "[9560, 9561].contains(term)"

hi,

I have a filtered query with facet. I want to get facets only for
explicit terms and avoid using size: X or all_terms (and filtering facets
on application side).

i've succeed to do that with 'script':

{'facets': {'category_ids': {'terms': {'field': 'category_ids',
'script': 'term == 9560'}}}}

the only thing i can't figure out is how to write mvel's script which
checks if term is in some list/array. Python example:

term in [1,2,3]

i've also tried to use python syntax adding 'lang' parameter with
values 'python'/'lang-python', but query failed with exception:
ElasticSearchIllegalArgumentException[script_lang not supported
[lang-python]]
or
ElasticSearchIllegalArgumentException[script_lang not supported
[python]]

any suggestions?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.