Error when searching multiple fields with different types

Elastic Stack Elasticsearch
1

I have been seeing strange error in my application and I was able to narrow it down and reproduce it in my simple test.

  1. A new index with 2 fields, one type long, another type string.
    curl -XPOST localhost:9200/test -d '{
    "mappings" : {
    "Cases" : {
    "properties" : {
    "case_number" : { "type" : "long"},
    "desc" : { "type" : "string"}
    }
    }
    }
    }'

  2. Index a record
    curl -XPUT 'http://localhost:9200/test/Cases/1' -d '{
    "case_number" : "123",
    "desc" : "abc"
    }'

  3. Search
    curl -XPOST "localhost:9200/test/Cases/_search?pretty=true" -d '{
    "query": {
    "query_string": {
    "fields" : ["case_number", "desc"],
    "query":"abc"
    }
    }
    }'

The search returns status 500 and produces this error:
"error" : "SearchPhaseExecutionException[Failed to execute phase [query_fetch], total failure; shardFailures {[tFucKjGMTjeW2EoWEcXDww][test][0]: SearchParseException[[test][0]: from[-1],size[-1]: Parse Failure [Failed to parse source [{\n "query": {\n "query_string": {\n "fields" : ["case_number", "desc"],\n "query":"abc"\n }\n }\n}]]]; nested: NumberFormatException[For input string: "abc"]; }]",

But if I replace my query "abc" with "123", the error goes away. But as you can see I have both string and long fields in my index so I would expect both from users.

Is this a bug or something I have to do to make ES handle both long and string types?

I am using the latest ES (0.19.2).

Thanks.

2

I don't think it's elasticsearch bug. Elasticsearch is just trying to
convert your query into a DisMax query (case_number:abc | desc:abc) and
case_number:abc just doesn't make sense here because case_number has type
long.

For your scenario to work, you need to index case_number as a string, or if
you need case_number as a long somewhere else (for sorting, for example),
index it as a multifield and include a string version into the user's query.

On Monday, April 9, 2012 1:25:04 PM UTC-4, chimingc wrote:

I have been seeing strange error in my application and I was able to narrow
it down and reproduce it in my simple test.

  1. A new index with 2 fields, one type long, another type string.
    curl -XPOST localhost:9200/test -d '{
    "mappings" : {
    "Cases" : {
    "properties" : {
    "case_number" : { "type" : "long"},
    "desc" : { "type" : "string"}
    }
    }
    }
    }'

  2. Index a record
    curl -XPUT 'http://localhost:9200/test/Cases/1' -d '{
    "case_number" : "123",
    "desc" : "abc"
    }'

  3. Search
    curl -XPOST "localhost:9200/test/Cases/_search?pretty=true" -d '{
    "query": {
    "query_string": {
    "fields" : ["case_number", "desc"],
    "query":"abc"
    }
    }
    }'

The search returns status 500 and produces this error:
"error" : "SearchPhaseExecutionException[Failed to execute phase
[query_fetch], total failure; shardFailures
{[tFucKjGMTjeW2EoWEcXDww][test][0]: SearchParseException[[test][0]:
from[-1],size[-1]: Parse Failure [Failed to parse source [{\n "query":
{\n "query_string": {\n "fields" : ["case_number",
"desc"],\n "query":"abc"\n }\n }\n}]]]; nested:
NumberFormatException[For input string: "abc"]; }]",

But if I replace my query "abc" with "123", the error goes away. But as you
can see I have both string and long fields in my index so I would expect
both from users.

Is this a bug or something I have to do to make ES handle both long and
string types?

I am using the latest ES (0.19.2).

Thanks.

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/Error-when-searching-multiple-fields-with-different-types-tp3897459p3897459.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

3

Thanks Igor.

Yes I agree case_number:abc does not make sense. But desc:abc does and has a hit so I would expect it to ignore case_number and return a hit instead of throwing an error.

I'd imagine that it's quite common to have multiple field types in one index. If so, how do people usually do? Or do people simply use strings for everything?

From: "Igor Motov-3 [via ElasticSearch Users]" <ml-node+s115913n3901461h95@n3.nabble.commailto:ml-node+s115913n3901461h95@n3.nabble.com>
Date: Tue, 10 Apr 2012 21:54:41 -0500
To: Jimmy Chen <jchen@sugarcrm.commailto:jchen@sugarcrm.com>
Subject: Re: Error when searching multiple fields with different types

I don't think it's elasticsearch bug. Elasticsearch is just trying to convert your query into a DisMax query (case_number:abc | desc:abc) and case_number:abc just doesn't make sense here because case_number has type long.

For your scenario to work, you need to index case_number as a string, or if you need case_number as a long somewhere else (for sorting, for example), index it as a multifield and include a string version into the user's query.

On Monday, April 9, 2012 1:25:04 PM UTC-4, chimingc wrote:
I have been seeing strange error in my application and I was able to narrow
it down and reproduce it in my simple test.

  1. A new index with 2 fields, one type long, another type string.
    curl -XPOST localhost:9200/test -d '{
    "mappings" : {
    "Cases" : {
    "properties" : {
    "case_number" : { "type" : "long"},
    "desc" : { "type" : "string"}
    }
    }
    }
    }'

  2. Index a record
    curl -XPUT 'http://localhost:9200/test/Cases/1' -d '{
    "case_number" : "123",
    "desc" : "abc"
    }'

  3. Search
    curl -XPOST "localhost:9200/test/Cases/_search?pretty=true" -d '{
    "query": {
    "query_string": {
    "fields" : ["case_number", "desc"],
    "query":"abc"
    }
    }
    }'

The search returns status 500 and produces this error:
"error" : "SearchPhaseExecutionException[Failed to execute phase
[query_fetch], total failure; shardFailures
{[tFucKjGMTjeW2EoWEcXDww][test][0]: SearchParseException[[test][0]:
from[-1],size[-1]: Parse Failure [Failed to parse source [{\n "query":
{\n "query_string": {\n "fields" : ["case_number",
"desc"],\n "query":"abc"\n }\n }\n}]]]; nested:
NumberFormatException[For input string: "abc"]; }]",

But if I replace my query "abc" with "123", the error goes away. But as you
can see I have both string and long fields in my index so I would expect
both from users.

Is this a bug or something I have to do to make ES handle both long and
string types?

I am using the latest ES (0.19.2).

Thanks.

--
View this message in context: http://elasticsearch-users.115913.n3.nabble.com/Error-when-searching-multiple-fields-with-different-types-tp3897459p3897459.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.

If you reply to this email, your message will be added to the discussion below:
http://elasticsearch-users.115913.n3.nabble.com/Error-when-searching-multiple-fields-with-different-types-tp3897459p3901461.html
To unsubscribe from Error when searching multiple fields with different types, click herehttp://elasticsearch-users.115913.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=3897459&code=amNoZW5Ac3VnYXJjcm0uY29tfDM4OTc0NTl8LTcxNDM5MzQ0Nw==.
NAMLhttp://elasticsearch-users.115913.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html!nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers!nabble%3Aemail.naml-instant_emails!nabble%3Aemail.naml-send_instant_email!nabble%3Aemail.naml

4

I think this is part of a larger discussion about what should search engine
do - ignore invalid user input or return an error. I can argue it both
ways. In most projects I worked on, we separated user query language from
search engine query language. It provided us a lot of flexibility with some
limited overhead. And when I translate user queries into search engine
queries, I prefer search engine returning error on invalid queries instead
of ignoring them, since it makes debugging process much easier.

In my opinion, when you use the same term to search multiple fields, it's
better to search fields with the same type. Otherwise, you might run into
issues like "why 12* finds 123 in description but doesn't find 123 in the
case number?" If you have a defined set of fields that you always want to
search by default, you can use _all fieldhttp://www.elasticsearch.org/guide/reference/mapping/all-field.html.
Elasticsearch provides a mechanism that allows you to control which fields
should be included there. If you have multiple sets of default fields that
you want to search in different situations, you can use multi field typehttp://www.elasticsearch.org/guide/reference/mapping/multi-field-type.htmlto index case number as a number (for sorting) and as a string (for
searching as part of a default field set).

On Thursday, April 12, 2012 1:14:18 PM UTC-4, chimingc wrote:

Thanks Igor.

Yes I agree case_number:abc does not make sense. But desc:abc does and has
a hit so I would expect it to ignore case_number and return a hit instead
of throwing an error.

I'd imagine that it's quite common to have multiple field types in one
index. If so, how do people usually do? Or do people simply use strings for
everything?

From: "Igor Motov-3 [via Elasticsearch Users]" <[hidden email]http://user/SendEmail.jtp?type=node&node=3905931&i=0

Date: Tue, 10 Apr 2012 21:54:41 -0500
To: Jimmy Chen <[hidden email]http://user/SendEmail.jtp?type=node&node=3905931&i=1

Subject: Re: Error when searching multiple fields with different types

I don't think it's elasticsearch bug. Elasticsearch is just trying to
convert your query into a DisMax query (case_number:abc | desc:abc) and
case_number:abc just doesn't make sense here because case_number has type
long.

For your scenario to work, you need to index case_number as a string, or
if you need case_number as a long somewhere else (for sorting, for
example), index it as a multifield and include a string version into the
user's query.

On Monday, April 9, 2012 1:25:04 PM UTC-4, chimingc wrote:

I have been seeing strange error in my application and I was able to
narrow
it down and reproduce it in my simple test.

  1. A new index with 2 fields, one type long, another type string.
    curl -XPOST localhost:9200/test -d '{
    "mappings" : {
    "Cases" : {
    "properties" : {
    "case_number" : { "type" : "long"},
    "desc" : { "type" : "string"}
    }
    }
    }
    }'

  2. Index a record
    curl -XPUT 'http://localhost:9200/test/Cases/1' -d '{
    "case_number" : "123",
    "desc" : "abc"
    }'

  3. Search
    curl -XPOST "localhost:9200/test/Cases/_search?pretty=true" -d '{
    "query": {
    "query_string": {
    "fields" : ["case_number", "desc"],
    "query":"abc"
    }
    }
    }'

The search returns status 500 and produces this error:
"error" : "SearchPhaseExecutionException[Failed to execute phase
[query_fetch], total failure; shardFailures
{[tFucKjGMTjeW2EoWEcXDww][test][0]: SearchParseException[[test][0]:
from[-1],size[-1]: Parse Failure [Failed to parse source [{\n "query":
{\n "query_string": {\n "fields" : ["case_number",
"desc"],\n "query":"abc"\n }\n }\n}]]]; nested:
NumberFormatException[For input string: "abc"]; }]",

But if I replace my query "abc" with "123", the error goes away. But as
you
can see I have both string and long fields in my index so I would expect
both from users.

Is this a bug or something I have to do to make ES handle both long and
string types?

I am using the latest ES (0.19.2).

Thanks.

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/Error-when-searching-multiple-fields-with-different-types-tp3897459p3897459.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

If you reply to this email, your message will be added to the discussion
below:

http://elasticsearch-users.115913.n3.nabble.com/Error-when-searching-multiple-fields-with-different-types-tp3897459p3901461.html
To unsubscribe from Error when searching multiple fields with different
types, click here.
NAMLhttp://elasticsearch-users.115913.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html!nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers!nabble%3Aemail.naml-instant_emails!nabble%3Aemail.naml-send_instant_email!nabble%3Aemail.naml

View this message in context: Re: Error when searching multiple fields
with different typeshttp://elasticsearch-users.115913.n3.nabble.com/Error-when-searching-multiple-fields-with-different-types-tp3897459p3905931.html
Sent from the Elasticsearch Users mailing list archivehttp://elasticsearch-users.115913.n3.nabble.com/at Nabble.com.