Field and query_string don't use configured analyzer

mattweber · February 15, 2012, 7:06pm

I have a field configured using an analyzer that does keyword tokenization and lowercasing. When I attempt to perform a search against this field using a field or query_string query, I don't get the expected result. Specifying the analyzer for the query does not help. If I do a text query against this field, everything works as expected. Am I doing something wrong?

My config, mapping, test document, and test queries can be found in this gist:

gist.github.com

https://gist.github.com/mattweber/1838124

config

index:
    analysis:
        analyzer:
            string_lowercase:
                tokenizer: keyword
                filter: lowercase

field query

# does not work
curl -XGET 'http://localhost:9200/test/type1/_search?pretty=true' -d '{
    "query":{
        "field":{
            "field1":{
                "query":"matt weber"
            }
        }
    }
}'

This file has been truncated. show original

mapping

curl -XPUT 'http://localhost:9200/test' -d '{
    "mappings" : {
        "type1" : {
            "properties" : {
                "field1" : {
                    "type" : "string",
                    "analyzer" : "string_lowercase"
                }
            }
        }

This file has been truncated. show original

There are more than three files. show original

Thanks!

--
Matt Weber
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)

mattweber · February 15, 2012, 7:26pm

Forgot to mention that I am using 0.19.0.RC2.

--
Matt Weber
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)

On Wednesday, February 15, 2012 at 11:06 AM, Matt Weber wrote:

I have a field configured using an analyzer that does keyword tokenization and lowercasing. When I attempt to perform a search against this field using a field or query_string query, I don't get the expected result. Specifying the analyzer for the query does not help. If I do a text query against this field, everything works as expected. Am I doing something wrong?

My config, mapping, test document, and test queries can be found in this gist:
config · GitHub

Thanks!

--
Matt Weber
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)

kimchy · February 15, 2012, 8:02pm

This happens because the Lucene query parser parses the text based on whitepsaces first in order to parse the syntax supported by it. So, the query ends up being on "matt" and "weber". You can wrap it in " characters to explicit state its a phrase, I commented on the gist with one that works.

The text query does not suffer it since it just takes the text and uses the relevant analyzer on it.

On Wednesday, February 15, 2012 at 9:06 PM, Matt Weber wrote:

I have a field configured using an analyzer that does keyword tokenization and lowercasing. When I attempt to perform a search against this field using a field or query_string query, I don't get the expected result. Specifying the analyzer for the query does not help. If I do a text query against this field, everything works as expected. Am I doing something wrong?

My config, mapping, test document, and test queries can be found in this gist:
config · GitHub

Thanks!

--
Matt Weber
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)

mattweber · February 15, 2012, 10:13pm

Works perfect, thank you! Something so simple…

--
Matt Weber
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)

On Wednesday, February 15, 2012 at 12:02 PM, Shay Banon wrote:

This happens because the Lucene query parser parses the text based on whitepsaces first in order to parse the syntax supported by it. So, the query ends up being on "matt" and "weber". You can wrap it in " characters to explicit state its a phrase, I commented on the gist with one that works.

The text query does not suffer it since it just takes the text and uses the relevant analyzer on it.

On Wednesday, February 15, 2012 at 9:06 PM, Matt Weber wrote:

I have a field configured using an analyzer that does keyword tokenization and lowercasing. When I attempt to perform a search against this field using a field or query_string query, I don't get the expected result. Specifying the analyzer for the query does not help. If I do a text query against this field, everything works as expected. Am I doing something wrong?

My config, mapping, test document, and test queries can be found in this gist:
config · GitHub

Thanks!

--
Matt Weber
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)