Query with script filter fails if query itself is filtered


(Jozsef Szalay) #1

I've been struggling with this issue for a while and I hope that someone
can help pointing me to a solution.

My setup is ES 1.0 on Linux with two ES nodes with 8GB memory allocated for
ES on each.

Here is how to recreate the issue:

curl -XPOST http://localhost:9200/my_index -d '{
"settings": {
"number_of_shards": 5
},
"mappings": {
"type1": {
"properties": {
"keys": {"type": "integer"}
}
},
"type2": {
"properties": {
"key": {"type": "string"},
"values": {"type": "integer"}
}
}
}
}'

curl -XPOST http://localhost:9200/my_index/_bulk -d '
{"index": {"_type": "type1", "_id": "a"}}
{"keys": [11,12,13]}
{"index": {"_type": "type2", "_id": "b"}}
{"key": "11", "values": [21,22]}
{"index": {"_type": "type2", "_id": "c"}}
{"key": "12", "values": [23,24]}
{"index": {"_type": "type2", "_id": "d"}}
{"key": "13", "values": [25,26]}
{"index": {"_type": "type2", "_id": "e"}}
{"key": "13", "values": [27,28]}
{"index": {"_type": "type2", "_id": "f"}}
{"key": "13", "values": [29,30]}
{"index": {"_type": "type2", "_id": "g"}}
{"key": "13", "values": [31,32]}
'

Note that I had to index more items than I actually search for. The error
doesn't show up otherwise.

Now, the following query reports an error in my environment:

curl -XGET http://localhost:9200/my_index/type2/_search?pretty=true -d '
{
"fields": [],
"query": {
"filtered": {
"filter": {
"terms": {
"key": {
"path": "keys",
"type": "type1",
"id": "a"
}
}
},
"query": {
"filtered": {
"filter": {
"script": {
"params": {
"v": 22
},
"script": "foreach(val : doc["values"].values) if
(val == v) return true; return false;"
}
}
}
}
}
}
}'

As you can see, a query is defined with a script filter and in turn that
query is filtered with a terms-lookup filter.
The results I'm getting show the following error:

{
"took" : 7,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 4,
"failed" : 1,
"failures" : [ {
"index" : "my_index",
"shard" : 0,
"status" : 500,
"reason" : "QueryPhaseExecutionException[[my_index][0]:
query[filtered(filtered(ConstantScore(ScriptFilter(foreach(val :
_source.values) if (val == v) return true; return false;)))->cache(key:11
key:12 key:13))->cache(_type:type2)],from[0],size[10]: Query Failed [Failed
to execute main query]]; nested: NullPointerException; "
} ]
},
"hits" : {
"total" : 1,
"max_score" : 1.0,
"hits" : [ {
"_index" : "my_index",
"_type" : "type2",
"_id" : "b",
"_score" : 1.0
} ]
}
}

The actual application that I'm working on with real data, but with similar
query reports either a NullPointerException or
"PropertyAccessException[[Error: could not access: ; in class:
org.elasticsearch.search.lookup.SourceLookup]" and the expected result set
is not returned.

The query above works if I change the script filter into a term filter.

curl -XGET http://localhost:9200/my_index/type2/_search?pretty=true -d '
{
"fields": [],
"query": {
"filtered": {
"filter": {
"terms": {
"key": {
"path": "keys",
"type": "type1",
"id": "a"
}
}
},
"query": {
"filtered": {
"filter": {
"term": {"values": 22}
}
}
}
}
}
}'

Unfortunately, I cannot replace the script filter with something simpler in
the real application.

One could argue, that the two filter sections could be combined to
something like this:

curl -XGET http://localhost:9200/my_index/type2/_search?pretty=true -d '
{
"fields": [],
"query": {
"filtered": {
"filter": {
"bool": {
"must": [
{"terms": {
"key": {
"path": "keys",
"type": "type1",
"id": "a"
}
}},
{"script": {
"params": {
"v": 22
},
"script": "foreach(val : doc["values"].values) if
(val == v) return true; return false;"
}}
]
}
}
}
}
}'

That appears to be working, and it may be the workaround for now, even
though I have to make some changes to the code that generates these
filters.
However, I still would like to understand what's happening here and whether
I stepped on a defect that should be addressed.

I created another post a while ago (herehttps://groups.google.com/forum/#!topic/elasticsearch/9tBf-IKnDkY)
that looks very similar and that may point to the same root cause.

Thanks for you help!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/45b9841d-7363-4479-8a8e-e6b11836fca4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(system) #2