Issue with query using named filter


(Andrei) #1

I'm trying to use the new named filters feature in 0.11 and getting a
QueryParsingException on this query:

{
"size": 10,
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"or": [
{
"geo_bounding_box": {
"geo": {
"top_left" : {
"lat" : 47.55,
"lon" : -124.10
},
"bottom_right" : {
"lat" : 31.06,
"lon" : -77
}
}
},
"_name": "foo"
},
{

                        "term": { "user_id": 11 },
                        "_name": "bar"
                }
            ]
        }
    }
}

}'

The trace is:

[20:36:30,713][DEBUG][action.search.type ] [Big Bertha]
[places_2010100101][0], node[cbecda40-810d-4f65-a163-4da95d9fe2dc],
[P], s[STARTED]: Failed to execute
[org.elasticsearch.action.search.SearchRequest@107525ad]
org.elasticsearch.search.SearchParseException: [places_2010100101][0]:
from[-1],size[10]: Parse Failure [Failed to parse [
{
"size": 10,
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"or": [
{
"geo_bounding_box": {
"geo": {
"top_left" : {
"lat" : 47.55,
"lon" : -124.10
},
"bottom_right" : {
"lat" : 31.06,
"lon" : -77
}
}
},
"_name": "foo"
},
{

                        "term": { "user_id": 11 },
                        "_name": "bar"
                }
            ]
        }
    }
}

}]]
at
org.elasticsearch.search.SearchService.parseSource(SearchService.java:
398)
at
org.elasticsearch.search.SearchService.createContext(SearchService.java:
311)
at
org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:
162)
at
org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:
131)
at
org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction
$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:
77)
at org.elasticsearch.action.search.type.TransportSearchTypeAction
$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:195)
at org.elasticsearch.action.search.type.TransportSearchTypeAction
$BaseAsyncAction.access$000(TransportSearchTypeAction.java:81)
at org.elasticsearch.action.search.type.TransportSearchTypeAction
$BaseAsyncAction$1.run(TransportSearchTypeAction.java:154)
at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:637)
Caused by: org.elasticsearch.index.query.QueryParsingException:
[places_2010100101] No query parser registered for [null]
at
org.elasticsearch.index.query.xcontent.QueryParseContext.parseInnerFilter(QueryParseContext.java:
168)
at
org.elasticsearch.index.query.xcontent.OrFilterParser.parse(OrFilterParser.java:
64)
at
org.elasticsearch.index.query.xcontent.QueryParseContext.parseInnerFilter(QueryParseContext.java:
170)
at
org.elasticsearch.index.query.xcontent.FilteredQueryParser.parse(FilteredQueryParser.java:
67)
at
org.elasticsearch.index.query.xcontent.QueryParseContext.parseInnerQuery(QueryParseContext.java:
144)
at
org.elasticsearch.index.query.xcontent.XContentIndexQueryParser.parse(XContentIndexQueryParser.java:
234)
at
org.elasticsearch.index.query.xcontent.XContentIndexQueryParser.parse(XContentIndexQueryParser.java:
214)
at
org.elasticsearch.search.query.QueryParseElement.parse(QueryParseElement.java:
34)
at
org.elasticsearch.search.SearchService.parseSource(SearchService.java:
385)
... 10 more


(Clinton Gormley) #2

On Fri, 2010-10-01 at 20:37 -0700, Andrei wrote:

I'm trying to use the new named filters feature in 0.11 and getting a
QueryParsingException on this query:

I haven't tried these yet, but looking at the issue


it seems that you should put the "_name" inside the filter:

{
"size": 10,
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"or": [
{
"geo_bounding_box": {
"geo": {
"top_left" : {
"lat" : 47.55,
"lon" : -124.10
},
"bottom_right" : {
"lat" : 31.06,
"lon" : -77
}
},
"_name": "foo"
}
},
{
"term": {
"user_id": 11,
"_name": "bar"
}
}
]
}
}
}
}'

clint


(Andrei) #3

That worked, thanks. Wish the error message was more helpful, though.

On Oct 1, 11:46 pm, Clinton Gormley clin...@iannounce.co.uk wrote:

On Fri, 2010-10-01 at 20:37 -0700, Andrei wrote:

I'm trying to use the new named filters feature in 0.11 and getting a
QueryParsingException on this query:

I haven't tried these yet, but looking at the issuehttp://github.com/elasticsearch/elasticsearch/issues/issue/364
it seems that you should put the "_name" inside the filter:

{
"size": 10,
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"or": [
{
"geo_bounding_box": {
"geo": {
"top_left" : {
"lat" : 47.55,
"lon" : -124.10
},
"bottom_right" : {
"lat" : 31.06,
"lon" : -77
}
},
"_name": "foo"
}
},
{
"term": {
"user_id": 11,
"_name": "bar"
}
}
]
}
}
}

}'

clint


(Andrei) #4

It seems that without the _name fields in the filters, the query runs
in < 0.01s, while with them it runs in about 2s. A drastic difference.
I only have about 120 documents in the index. Is there an explanation
for this?

time curl -XPOST "http://localhost:9200/places/place/_search?
pretty=true" -d'
{
"size": 10,
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"or": [
{
"geo_bounding_box": {
"geo": {
"top_left" : {
"lat" : 47.55,
"lon" : -124.10
},
"bottom_right" : {
"lat" : 31.06,
"lon" : -77
}
},
"_name": "geo_filter"
}
},
{
"term": {
"user_id": 11,
"_name": "user_filter"
}
}
]
}
}
}
}'

On Oct 2, 12:21 am, Andrei and...@zmievski.org wrote:

That worked, thanks. Wish the error message was more helpful, though.


(Shay Banon) #5

There should not be that big different, can you post a test that fills
elasticsearch with data and recreates it? Note that the first time executing
it might take time, second executions should be faster.

-shay.banon

On Sat, Oct 2, 2010 at 9:25 AM, Andrei andrei@zmievski.org wrote:

It seems that without the _name fields in the filters, the query runs
in < 0.01s, while with them it runs in about 2s. A drastic difference.
I only have about 120 documents in the index. Is there an explanation
for this?

time curl -XPOST "http://localhost:9200/places/place/_search?
pretty=true" -d'
{
"size": 10,
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"or": [
{
"geo_bounding_box": {
"geo": {
"top_left" : {
"lat" : 47.55,
"lon" : -124.10
},
"bottom_right" : {
"lat" : 31.06,
"lon" : -77
}
},
"_name": "geo_filter"
}
},
{
"term": {
"user_id": 11,
"_name": "user_filter"
}
}
]
}
}
}
}'

On Oct 2, 12:21 am, Andrei and...@zmievski.org wrote:

That worked, thanks. Wish the error message was more helpful, though.


(Shay Banon) #6

By first time, I meant first time something "touches" the lat/lon values, it
will cause them to be loaded, and then executions on them will be fast. Post
that, same filter execution will also be faster because filter results are
further cached.

On Sat, Oct 2, 2010 at 2:45 PM, Shay Banon shay.banon@elasticsearch.comwrote:

There should not be that big different, can you post a test that fills
elasticsearch with data and recreates it? Note that the first time executing
it might take time, second executions should be faster.

-shay.banon

On Sat, Oct 2, 2010 at 9:25 AM, Andrei andrei@zmievski.org wrote:

It seems that without the _name fields in the filters, the query runs
in < 0.01s, while with them it runs in about 2s. A drastic difference.
I only have about 120 documents in the index. Is there an explanation
for this?

time curl -XPOST "http://localhost:9200/places/place/_search?
pretty=true" -d'
{
"size": 10,
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"or": [
{
"geo_bounding_box": {
"geo": {
"top_left" : {
"lat" : 47.55,
"lon" : -124.10
},
"bottom_right" : {
"lat" : 31.06,
"lon" : -77
}
},
"_name": "geo_filter"
}
},
{
"term": {
"user_id": 11,
"_name": "user_filter"
}
}
]
}
}
}
}'

On Oct 2, 12:21 am, Andrei and...@zmievski.org wrote:

That worked, thanks. Wish the error message was more helpful, though.


(Andrei) #7

I posted the test file that contains a sequence of curl commands that
creates and populates the index and does the search with and without
named filters.

http://zmievski.org/dump/es11-test.zip

–Andrei

On Oct 2, 5:45 am, Shay Banon shay.ba...@elasticsearch.com wrote:

There should not be that big different, can you post a test that fills
elasticsearch with data and recreates it? Note that the first time executing
it might take time, second executions should be faster.

-shay.banon


(Shay Banon) #8

Hey,

That little extra data seems to have hit the curl problem discussed in
another thread:
http://groups.google.com/a/elasticsearch.com/group/users/browse_thread/thread/c99db77c9f92af99#.
If you add to the curl command: -H 'Expect:', then it will run as fast as it
should be.

I am trying to see if its possible to support this expected behavior from
curl, but the http lib I use does not have a nice extension point for
handling it, so working on a patch for them. In any case, that extra
chitchat is better disabled even if there is support for this curl
behavior.

-shay.banon

On Sun, Oct 3, 2010 at 2:24 AM, Andrei andrei@zmievski.org wrote:

I posted the test file that contains a sequence of curl commands that
creates and populates the index and does the search with and without
named filters.

http://zmievski.org/dump/es11-test.zip

–Andrei

On Oct 2, 5:45 am, Shay Banon shay.ba...@elasticsearch.com wrote:

There should not be that big different, can you post a test that fills
elasticsearch with data and recreates it? Note that the first time
executing
it might take time, second executions should be faster.

-shay.banon


(Andrei) #9

Ran into another snag, where I can't seem to attach a _name field to
an "and" filter inside an "or" filter:

{
"size": 10,
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"or": [
{
"term": { "user_id": 11, "_name": "user" }
},
{
"and": [
{ "terms": { "user_id":
[19,23,100,101] } },
{ "range": { "privacy": { "gt": 0 } } }
],
"_name": "contacts"
}
]
}
}
}
}

This gives me:

...
Caused by: org.elasticsearch.index.query.QueryParsingException:
[places_2010100101] [or] filter requires 'filters' to be set on it'
...

Where should the _name field go in that "and" filter?

–Andrei

On Oct 1, 11:46 pm, Clinton Gormley clin...@iannounce.co.uk wrote:

On Fri, 2010-10-01 at 20:37 -0700, Andrei wrote:

I'm trying to use the new named filters feature in 0.11 and getting a
QueryParsingException on this query:

I haven't tried these yet, but looking at the issuehttp://github.com/elasticsearch/elasticsearch/issues/issue/364
it seems that you should put the "_name" inside the filter:

{
"size": 10,
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"or": [
{
"geo_bounding_box": {
"geo": {
"top_left" : {
"lat" : 47.55,
"lon" : -124.10
},
"bottom_right" : {
"lat" : 31.06,
"lon" : -77
}
},
"_name": "foo"
}
},
{
"term": {
"user_id": 11,
"_name": "bar"
}
}
]
}
}
}

}'

clint


(Shay Banon) #10

With an and filter, you need to put the filters under a filter field (as
an array), and then provide the _name there, for example:

"and" : {
"filters" : [
{ "term" : .. },
{ "term" ...}
],
"_name" : "the filter name"
}

On Mon, Oct 4, 2010 at 3:07 AM, Andrei andrei@zmievski.org wrote:

Ran into another snag, where I can't seem to attach a _name field to
an "and" filter inside an "or" filter:

{
"size": 10,
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"or": [
{
"term": { "user_id": 11, "_name": "user" }
},
{
"and": [
{ "terms": { "user_id":
[19,23,100,101] } },
{ "range": { "privacy": { "gt": 0 } } }
],
"_name": "contacts"
}
]
}
}
}
}

This gives me:

...
Caused by: org.elasticsearch.index.query.QueryParsingException:
[places_2010100101] [or] filter requires 'filters' to be set on it'
...

Where should the _name field go in that "and" filter?

–Andrei

On Oct 1, 11:46 pm, Clinton Gormley clin...@iannounce.co.uk wrote:

On Fri, 2010-10-01 at 20:37 -0700, Andrei wrote:

I'm trying to use the new named filters feature in 0.11 and getting a
QueryParsingException on this query:

I haven't tried these yet, but looking at the issuehttp://
github.com/elasticsearch/elasticsearch/issues/issue/364
it seems that you should put the "_name" inside the filter:

{
"size": 10,
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"or": [
{
"geo_bounding_box": {
"geo": {
"top_left" : {
"lat" : 47.55,
"lon" : -124.10
},
"bottom_right" : {
"lat" : 31.06,
"lon" : -77
}
},
"_name": "foo"
}
},
{
"term": {
"user_id": 11,
"_name": "bar"
}
}
]
}
}
}

}'

clint


(system) #11