Querying "Hash" Values Stored under String Mapping


(govind201-2) #1

My index has a field called "features" with mapping type "string" as
follows:

{

"twitter": {

    "user": {

        "properties": {

            "features": {

                "type": "string"

            }

        }

    }

}

}

Here are three examples of potential values that may be stored in
"features":

Example 1: name bob age thirty

Example 2: ["name", "bob", "age", "thirty"]

Example 3: {"name" : "bob", "age" : "thirty"}

The query

curl -XGET http://localhost:9200/twitter/user/_search -d '{ "query" : {
"text" : { "features" : "age" } } }'

returns examples 1 and 2, but not example 3.

The "features" field may contain any number of keys (such as age, name,
fieldX, fieldYZ …) and I want to maintain a strict mapping, hence my choice
of the "string" type mapping. I do, however, want to carry out text
searches along the words, i.e. return example 3 if the query is either
"name", "bob", "age" or "thirty". Is there any query that can return the
results I seek?

FYI, running the analyze API along example 3:

curl -XGET "localhost:9200/twitter/_analyze?tokenizer=standard&pretty=true"
-d '{"name" : "bob", "age" : "thirty"}'

seems to tokenize the string as expected into "thirty", "age", "bob" and
"name".

Thanks!

--


(Radu Gheorghe) #2

Hello,

Maybe I'm missing something, because for me it works. Here's what I did:

===================================

curl localhost:9200/testing/test/_mapping?pretty=true

{
"test" : {
"properties" : {
"features" : {
"type" : "string"
}
}
}
}

curl -XPOST localhost:9200/testing/test -d '{

"features": "{\"name\" : \"bob\", \"age\" : \"thirty\"}"

}'
{"ok":true,"_index":"testing","_type":"test","_id":"N974zlY6T6KwSWokv3n-1g","_version":1}

curl http://localhost:9200/testing/test/_search?pretty=true -d '{

"query" : { "text" : { "features" : "age" } } }'
{
<--- REMOVED SOME STUFF HERE --->
"hits" : [ {
"_index" : "testing",
"_type" : "test",
"_id" : "N974zlY6T6KwSWokv3n-1g",
"_score" : 0.15342641, "_source" : {
"features": "{"name" : "bob", "age" : "thirty"}"
}
} ]
}
}

Which version of Elasticsearch are you using?

Best regards,
Radu

http://sematext.com/ -- ElasticSearch -- Solr -- Lucene

On Tue, Oct 23, 2012 at 11:32 AM, govind201 govind@semantics3.com wrote:

My index has a field called "features" with mapping type "string" as
follows:

{

"twitter": {

    "user": {

        "properties": {

            "features": {

                "type": "string"

            }

        }

    }

}

}

Here are three examples of potential values that may be stored in
"features":

Example 1: name bob age thirty

Example 2: ["name", "bob", "age", "thirty"]

Example 3: {"name" : "bob", "age" : "thirty"}

The query

curl -XGET http://localhost:9200/twitter/user/_search -d '{ "query" : {
"text" : { "features" : "age" } } }'

returns examples 1 and 2, but not example 3.

The "features" field may contain any number of keys (such as age, name,
fieldX, fieldYZ …) and I want to maintain a strict mapping, hence my choice
of the "string" type mapping. I do, however, want to carry out text searches
along the words, i.e. return example 3 if the query is either "name", "bob",
"age" or "thirty". Is there any query that can return the results I seek?

FYI, running the analyze API along example 3:

curl -XGET "localhost:9200/twitter/_analyze?tokenizer=standard&pretty=true"
-d '{"name" : "bob", "age" : "thirty"}'

seems to tokenize the string as expected into "thirty", "age", "bob" and
"name".

Thanks!

--


(simonw-2) #3

On Tuesday, October 23, 2012 10:32:09 AM UTC+2, govind201 wrote:

My index has a field called "features" with mapping type "string" as
follows:

{

"twitter": {

    "user": {

        "properties": {

            "features": {

                "type": "string"

            }

        }

    }

}

}

Here are three examples of potential values that may be stored in
"features":

Example 1: name bob age thirty

Example 2: ["name", "bob", "age", "thirty"]

Example 3: {"name" : "bob", "age" : "thirty"}

hey, are you escaping the hash correctly? it should be astring value right?

simon

The query

curl -XGET http://localhost:9200/twitter/user/_search -d '{ "query" : {
"text" : { "features" : "age" } } }'

returns examples 1 and 2, but not example 3.

The "features" field may contain any number of keys (such as age, name,
fieldX, fieldYZ …) and I want to maintain a strict mapping, hence my choice
of the "string" type mapping. I do, however, want to carry out text
searches along the words, i.e. return example 3 if the query is either
"name", "bob", "age" or "thirty". Is there any query that can return the
results I seek?

FYI, running the analyze API along example 3:

curl -XGET
"localhost:9200/twitter/_analyze?tokenizer=standard&pretty=true" -d
'{"name" : "bob", "age" : "thirty"}'

seems to tokenize the string as expected into "thirty", "age", "bob" and
"name".

Thanks!

--


(govind201-2) #4

I haven't escaped my hash as Radu has done in his example. The data,
however, seems to have indexed successfully even without escaping.

Here are three examples:
curl -XPUT http://localhost:9200/twitter/user/1 -d '{ "features" : "name
bob age thirty"}'
curl -XPUT http://localhost:9200/twitter/user/2 -d '{ "features" : {"name"
: "bob", "age" : "thirty"}}'
curl -XPUT http://localhost:9200/twitter/user/3 -d '{ "features" :
"{"name" : "bob", "age" : "thirty"}}'

Here's what I get when I run a full search:
curl -XGET localhost:9200/insta/_search
"hits": [
{
"_id": "1",
"_index": "insta",
"_score": 1.0,
"_source": {
"features": "name bob age thirty"
},
"_type": "user"
},
{
"_id": "2",
"_index": "insta",
"_score": 1.0,
"_source": {
"features": {
"age": "thirty",
"name": "bob"
}
},
"_type": "user"
},
{
"_id": "3",
"_index": "insta",
"_score": 1.0,
"_source": {
"features": "{"name" : "bob", "age" : "thirty"}"
},
"_type": "user"
}
]

And here's what I get with the text search:
"hits" : [ {
"_index" : "insta",
"_type" : "user",
"_id" : "1",
"_score" : 0.15342641, "_source" : { "features" : "name bob age
thirty"}
}, {
"_index" : "insta",
"_type" : "user",
"_id" : "3",
"_score" : 0.15342641, "_source" : { "features": "{"name" :
"bob", "age" : "thirty"}" }
} ]

I didn't think there would be a difference because the analyze API returns
the same results for both of the examples below:
curl -XGET "localhost:9200/twitter/_analyze?tokenizer=standard&pretty=true"
-d '{"name" : "bob", "age" : "thirty"}'
curl -XGET "localhost:9200/twitter/_analyze?tokenizer=standard&pretty=true"
-d '{"name" : "bob", "age" : "thirty"}}'

I'm curious to know how id 2 is stored internally. Also, I'll be sure to
escape strings the hash in the future (thanks for that all), but i'll need
to reindex a lot of data - is there any other to run text searches with
data indexed in the manner shown for id 2?

Thanks!

On Tuesday, October 23, 2012 5:20:20 PM UTC+8, simonw wrote:

On Tuesday, October 23, 2012 10:32:09 AM UTC+2, govind201 wrote:

My index has a field called "features" with mapping type "string" as
follows:

{

"twitter": {

    "user": {

        "properties": {

            "features": {

                "type": "string"

            }

        }

    }

}

}

Here are three examples of potential values that may be stored in
"features":

Example 1: name bob age thirty

Example 2: ["name", "bob", "age", "thirty"]

Example 3: {"name" : "bob", "age" : "thirty"}

hey, are you escaping the hash correctly? it should be astring value
right?

simon

The query

curl -XGET http://localhost:9200/twitter/user/_search -d '{ "query" : {
"text" : { "features" : "age" } } }'

returns examples 1 and 2, but not example 3.

The "features" field may contain any number of keys (such as age, name,
fieldX, fieldYZ …) and I want to maintain a strict mapping, hence my choice
of the "string" type mapping. I do, however, want to carry out text
searches along the words, i.e. return example 3 if the query is either
"name", "bob", "age" or "thirty". Is there any query that can return the
results I seek?

FYI, running the analyze API along example 3:

curl -XGET
"localhost:9200/twitter/_analyze?tokenizer=standard&pretty=true" -d
'{"name" : "bob", "age" : "thirty"}'

seems to tokenize the string as expected into "thirty", "age", "bob" and
"name".

Thanks!

--


(Zhibin) #5

{
"_id": "2",
"_index": "insta",
"_score": 1.0,
"_source": {
"features": {
"age": "thirty",
"name": "bob"
}
},
"_type": "user"
},

As you did not escape the quotation marks, the parser reads as it under
features, {field=age, value=thirty}, {field=name, value=bob}.
Your text search is looking for value of "age", thus will not match as it
is a field name.

Note in 0.19.9, text query is renamed to match queryhttp://www.elasticsearch.org/guide/reference/query-dsl/match-query.html
.

--


(govind201-2) #6

Right, so is there any way at all to query the text now, even if it be by
age or name ("features.age"/"features.name" don't work)? After all, mapping
for the field is of type "string".

On Wednesday, October 24, 2012 5:00:07 PM UTC+8, Zhibin wrote:

{
"_id": "2",
"_index": "insta",
"_score": 1.0,
"_source": {
"features": {
"age": "thirty",
"name": "bob"
}
},
"_type": "user"
},

As you did not escape the quotation marks, the parser reads as it under
features, {field=age, value=thirty}, {field=name, value=bob}.
Your text search is looking for value of "age", thus will not match as it
is a field name.

Note in 0.19.9, text query is renamed to match queryhttp://www.elasticsearch.org/guide/reference/query-dsl/match-query.html
.

--


(Radu Gheorghe) #7

Hello,

Now that "name" and "age" are fields, you can either query for the
content of those fields, like:

curl localhost:9200/twitter/user/_search?pretty=true -d '{
"query": {
"match": {
"features.age": "thirty"
}
}
}'

By default, they're included in _all, so they would appear in this as
well, but you might get unwanted results from other fields:

curl localhost:9200/twitter/user/_search?pretty=true -d '{
"query": {
"match": {
"_all": "thirty"
}
}
}'

And you can also check for the existence of fields like "age" with filters:

curl localhost:9200/twitter/user/_search?pretty=true -d '{
"query": {
"constant_score" : {
"filter" : {
"exists" : { "field" : "features.age" }
}
}
}
}'

Best regards,
Radu

http://sematext.com/ -- ElasticSearch -- Solr -- Lucene

On Wed, Oct 24, 2012 at 12:27 PM, govind201 govind@semantics3.com wrote:

Right, so is there any way at all to query the text now, even if it be by
age or name ("features.age"/"features.name" don't work)? After all, mapping
for the field is of type "string".

On Wednesday, October 24, 2012 5:00:07 PM UTC+8, Zhibin wrote:

{
"_id": "2",
"_index": "insta",
"_score": 1.0,
"_source": {
"features": {
"age": "thirty",
"name": "bob"
}
},
"_type": "user"
},

As you did not escape the quotation marks, the parser reads as it under
features, {field=age, value=thirty}, {field=name, value=bob}.
Your text search is looking for value of "age", thus will not match as it
is a field name.

Note in 0.19.9, text query is renamed to match query.

--

--


(govind201-2) #8

Hi Radu,
I didn't get the results expected for any of the queries that you
mentioned. I even upgraded from 0.19.8 to 0.19.11 just in case. That query
behavior seems logical for your standard case, but in my case, the mapping
is of type "string", as I'd mentioned:

{

"twitter": {

    "user": {

        "properties": {

            "features": {

                "type": "string"

            }

        }

    }

}

}

On Wednesday, October 24, 2012 10:02:39 PM UTC+8, Radu Gheorghe wrote:

Hello,

Now that "name" and "age" are fields, you can either query for the
content of those fields, like:

curl localhost:9200/twitter/user/_search?pretty=true -d '{
"query": {
"match": {
"features.age": "thirty"
}
}
}'

By default, they're included in _all, so they would appear in this as
well, but you might get unwanted results from other fields:

curl localhost:9200/twitter/user/_search?pretty=true -d '{
"query": {
"match": {
"_all": "thirty"
}
}
}'

And you can also check for the existence of fields like "age" with
filters:

curl localhost:9200/twitter/user/_search?pretty=true -d '{
"query": {
"constant_score" : {
"filter" : {
"exists" : { "field" : "features.age" }
}
}
}
}'

Best regards,
Radu

http://sematext.com/ -- ElasticSearch -- Solr -- Lucene

On Wed, Oct 24, 2012 at 12:27 PM, govind201 <gov...@semantics3.com<javascript:>>
wrote:

Right, so is there any way at all to query the text now, even if it be
by
age or name ("features.age"/"features.name" don't work)? After all,
mapping
for the field is of type "string".

On Wednesday, October 24, 2012 5:00:07 PM UTC+8, Zhibin wrote:

{
"_id": "2",
"_index": "insta",
"_score": 1.0,
"_source": {
"features": {
"age": "thirty",
"name": "bob"
}
},
"_type": "user"
},

As you did not escape the quotation marks, the parser reads as it under
features, {field=age, value=thirty}, {field=name, value=bob}.
Your text search is looking for value of "age", thus will not match as
it

is a field name.

Note in 0.19.9, text query is renamed to match query.

--

--


(Radu Gheorghe) #9

Hello,

I think I understand now. So if your mapping is string, like this:

"test" : {
"properties" : {
"foo" : {
"type" : "string"
}
}
}

And if you index an unescaped JSON, like this:

curl -XPUT localhost:9200/test/test/111 -d '{"foo":{"bar":"baz"}}'

You should end up only with the words, like this:

"_source" : {"foo":"bar baz"}

So your field names within the string, like "bar" in the example
above, would show up in a match query, like this:

curl -XPOST localhost:9200/test/test/_search?pretty=true -d '{
"query": {
"match": {
"foo": "bar"
}
}
}'

Best regards,
Radu

http://sematext.com/ -- ElasticSearch -- Solr -- Lucene

On Wed, Oct 24, 2012 at 8:23 PM, govind201 govind@semantics3.com wrote:

Hi Radu,
I didn't get the results expected for any of the queries that you mentioned.
I even upgraded from 0.19.8 to 0.19.11 just in case. That query behavior
seems logical for your standard case, but in my case, the mapping is of type
"string", as I'd mentioned:

{

"twitter": {

    "user": {

        "properties": {

            "features": {

                "type": "string"

            }

        }

    }

}

}

On Wednesday, October 24, 2012 10:02:39 PM UTC+8, Radu Gheorghe wrote:

Hello,

Now that "name" and "age" are fields, you can either query for the
content of those fields, like:

curl localhost:9200/twitter/user/_search?pretty=true -d '{
"query": {
"match": {
"features.age": "thirty"
}
}
}'

By default, they're included in _all, so they would appear in this as
well, but you might get unwanted results from other fields:

curl localhost:9200/twitter/user/_search?pretty=true -d '{
"query": {
"match": {
"_all": "thirty"
}
}
}'

And you can also check for the existence of fields like "age" with
filters:

curl localhost:9200/twitter/user/_search?pretty=true -d '{
"query": {
"constant_score" : {
"filter" : {
"exists" : { "field" : "features.age" }
}
}
}
}'

Best regards,
Radu

http://sematext.com/ -- ElasticSearch -- Solr -- Lucene

On Wed, Oct 24, 2012 at 12:27 PM, govind201 gov...@semantics3.com wrote:

Right, so is there any way at all to query the text now, even if it be
by
age or name ("features.age"/"features.name" don't work)? After all,
mapping
for the field is of type "string".

On Wednesday, October 24, 2012 5:00:07 PM UTC+8, Zhibin wrote:

{
"_id": "2",
"_index": "insta",
"_score": 1.0,
"_source": {
"features": {
"age": "thirty",
"name": "bob"
}
},
"_type": "user"
},

As you did not escape the quotation marks, the parser reads as it under
features, {field=age, value=thirty}, {field=name, value=bob}.
Your text search is looking for value of "age", thus will not match as
it
is a field name.

Note in 0.19.9, text query is renamed to match query.

--

--

--


(govind201-2) #10

That's exactly the behavior that I'd sought. But the query doesn't work as
you describe it, i.e., it returns no hits. Try these one after the other
and you'll see:
curl -XPUT localhost:9200/test
curl -XPUT 'localhost:9200/test/test/_mapping' -d ' { "test" : {
"properties" : { "foo" : {"type" : "string"} } } } '
curl -XPUT localhost:9200/test/test/2 -d '{"foo":{"bar":"baz"}}'
curl -XPOST localhost:9200/test/test/_search?pretty=true -d '{ "query": {
"match": { "foo": "bar" } } }'

I can't seem to retrieve document 2 for any sort of match query. It is,
however, returned when I execute:
curl -XPOST localhost:9200/test/test/_search?pretty=true

On Thursday, October 25, 2012 4:25:14 AM UTC+8, Radu Gheorghe wrote:

Hello,

I think I understand now. So if your mapping is string, like this:

"test" : {
"properties" : {
"foo" : {
"type" : "string"
}
}
}

And if you index an unescaped JSON, like this:

curl -XPUT localhost:9200/test/test/111 -d '{"foo":{"bar":"baz"}}'

You should end up only with the words, like this:

"_source" : {"foo":"bar baz"}

So your field names within the string, like "bar" in the example
above, would show up in a match query, like this:

curl -XPOST localhost:9200/test/test/_search?pretty=true -d '{
"query": {
"match": {
"foo": "bar"
}
}
}'

Best regards,
Radu

http://sematext.com/ -- ElasticSearch -- Solr -- Lucene

On Wed, Oct 24, 2012 at 8:23 PM, govind201 <gov...@semantics3.com<javascript:>>
wrote:

Hi Radu,
I didn't get the results expected for any of the queries that you
mentioned.
I even upgraded from 0.19.8 to 0.19.11 just in case. That query behavior
seems logical for your standard case, but in my case, the mapping is of
type
"string", as I'd mentioned:

{

"twitter": { 

    "user": { 

        "properties": { 

            "features": { 

                "type": "string" 

            } 

        } 

    } 

} 

}

On Wednesday, October 24, 2012 10:02:39 PM UTC+8, Radu Gheorghe wrote:

Hello,

Now that "name" and "age" are fields, you can either query for the
content of those fields, like:

curl localhost:9200/twitter/user/_search?pretty=true -d '{
"query": {
"match": {
"features.age": "thirty"
}
}
}'

By default, they're included in _all, so they would appear in this as
well, but you might get unwanted results from other fields:

curl localhost:9200/twitter/user/_search?pretty=true -d '{
"query": {
"match": {
"_all": "thirty"
}
}
}'

And you can also check for the existence of fields like "age" with
filters:

curl localhost:9200/twitter/user/_search?pretty=true -d '{
"query": {
"constant_score" : {
"filter" : {
"exists" : { "field" : "features.age" }
}
}
}
}'

Best regards,
Radu

http://sematext.com/ -- ElasticSearch -- Solr -- Lucene

On Wed, Oct 24, 2012 at 12:27 PM, govind201 gov...@semantics3.com
wrote:

Right, so is there any way at all to query the text now, even if it
be

by
age or name ("features.age"/"features.name" don't work)? After all,
mapping
for the field is of type "string".

On Wednesday, October 24, 2012 5:00:07 PM UTC+8, Zhibin wrote:

{
"_id": "2",
"_index": "insta",
"_score": 1.0,
"_source": {
"features": {
"age": "thirty",
"name": "bob"
}
},
"_type": "user"
},

As you did not escape the quotation marks, the parser reads as it
under

features, {field=age, value=thirty}, {field=name, value=bob}.
Your text search is looking for value of "age", thus will not match
as

it
is a field name.

Note in 0.19.9, text query is renamed to match query.

--

--

--


(Clinton Gormley) #11

Hiya

"test" : {
"properties" : {
"foo" : {
"type" : "string"
}
}
}

And if you index an unescaped JSON, like this:

curl -XPUT localhost:9200/test/test/111 -d '{"foo":{"bar":"baz"}}'

You should end up only with the words, like this:

"_source" : {"foo":"bar baz"}

This wouldn't work. ES receives field 'foo' which is of type 'object',
compares it to the mapping and finds that field 'foo' should be of type
'string', so it ignores it.

If you want the bar/baz json to be indexed as text, then you need to
pass it as text:

curl -XPOST 'http://127.0.0.1:9200/test/test?pretty=1' -d '
{
"foo" : "{"bar":"baz"}"
}
'

clint

--


(Radu Gheorghe) #12

Hello,

Now I'm both curious and confused about this behavior.

So yesterday I tested before writing here what's described below and
it worked. I just copy-pasted here. Now, I can't reproduce the same
behavior (went through the history - I still don't get it). Instead, I
get what govind201 reported. I was using 0.19.11 all this time.

Furthermore, the document "_source" : {"foo":{"bar":"baz"}} doesn't
appear with any of the following:

'{"filter":{"exists":{"field":"foo"}}}'
'{"filter":{"exists":{"field":"_all"}}}'

But it appears on this:

'{"filter":{"script":{"script":"_source.foo.bar == "baz""}}}'

So what I understand up to this point is that the document is stored,
but not indexed. That's what you meant, Clint?

That said, it seems to be fine if I throw in an integer there instead
of a string:

curl -XPUT localhost:9200/test/test/111 -d '{"foo":2}'

curl -XPOST localhost:9200/test/test/_search?pretty=true -d '{

"query": { "match": { "foo":2}}}'

curl -XPOST localhost:9200/test/test/_search?pretty=true -d

'{"filter":{"range":{"foo":{"from": 1, "to": 3}}}}'

Back to the topic, I think the best thing that can be done here is to
reindex your data, this time passing strings as strings.

Best regards,
Radu

http://sematext.com/ -- ElasticSearch -- Solr -- Lucene

On Thu, Oct 25, 2012 at 11:27 AM, Clinton Gormley clint@traveljury.com wrote:

Hiya

"test" : {
"properties" : {
"foo" : {
"type" : "string"
}
}
}

And if you index an unescaped JSON, like this:

curl -XPUT localhost:9200/test/test/111 -d '{"foo":{"bar":"baz"}}'

You should end up only with the words, like this:

"_source" : {"foo":"bar baz"}

This wouldn't work. ES receives field 'foo' which is of type 'object',
compares it to the mapping and finds that field 'foo' should be of type
'string', so it ignores it.

If you want the bar/baz json to be indexed as text, then you need to
pass it as text:

curl -XPOST 'http://127.0.0.1:9200/test/test?pretty=1' -d '
{
"foo" : "{"bar":"baz"}"
}
'

clint

--

--


(Clinton Gormley) #13

Hi Radu

So yesterday I tested before writing here what's described below and
it worked. I just copy-pasted here. Now, I can't reproduce the same
behavior (went through the history - I still don't get it). Instead, I
get what govind201 reported. I was using 0.19.11 all this time.

I think you probably deleted the index then indexed the doc, which
created a mapping for the 'foo' field of type 'object'.

Furthermore, the document "_source" : {"foo":{"bar":"baz"}} doesn't
appear with any of the following:

'{"filter":{"exists":{"field":"foo"}}}'
'{"filter":{"exists":{"field":"_all"}}}'

Correct. Because the {bar:baz} value for foo is ignored, the 'foo'
field is "missing", and similarly, no values have been passed to '_all',
so that is also "missing".

But it appears on this:

'{"filter":{"script":{"script":"_source.foo.bar == "baz""}}}'

The _source field just returns whatever JSON doc you passed to
elasticsearch. It has nothing to do with what is indexed.

So what I understand up to this point is that the document is stored,
but not indexed. That's what you meant, Clint?

Correct

That said, it seems to be fine if I throw in an integer there instead
of a string:

curl -XPUT localhost:9200/test/test/111 -d '{"foo":2}'

curl -XPOST localhost:9200/test/test/_search?pretty=true -d '{

"query": { "match": { "foo":2}}}'

curl -XPOST localhost:9200/test/test/_search?pretty=true -d

'{"filter":{"range":{"foo":{"from": 1, "to": 3}}}}'

Yes, value 2 becomes "2" because it is type 'string'. Note, however,
that your range is string based, not number based.

curl -XPOST 'http://127.0.0.1:9200/test/test' -d '{"foo": 2}'
curl -XPOST 'http://127.0.0.1:9200/test/test' -d '{"foo": 20}'
curl -XPOST 'http://127.0.0.1:9200/test/test' -d '{"foo": 3}'

curl -XGET 'http://127.0.0.1:9200/test/test/_search?pretty=1' -d '
{
"query" : {
"range" : {
"foo" : {
"from" : 2,
"to" : 3
}
}
}
}
'

This will return docs '2' and '20' but not '3'

"hits" : [

{

"_source" : {

"foo" : 2

},

"_score" : 1,

"_index" : "test",

"_id" : "SsfC4j-5RpuEPfg4FEPvzQ",

"_type" : "test"

},

{

"_source" : {

"foo" : 20

},

"_score" : 1,

"_index" : "test",

"_id" : "Du9X5JfQSTu4Jljotg8MFw",

"_type" : "test"

}

],

clint

--


(Radu Gheorghe) #14

Hi Clinton,

On Thu, Oct 25, 2012 at 12:26 PM, Clinton Gormley clint@traveljury.com wrote:

Hi Radu

So yesterday I tested before writing here what's described below and
it worked. I just copy-pasted here. Now, I can't reproduce the same
behavior (went through the history - I still don't get it). Instead, I
get what govind201 reported. I was using 0.19.11 all this time.

I think you probably deleted the index then indexed the doc, which
created a mapping for the 'foo' field of type 'object'.

Neah, I checked through the history and I didn't do that. Plus, it
couldn't have ended up as "foo": "bar baz" if I did that, right?

Furthermore, the document "_source" : {"foo":{"bar":"baz"}} doesn't
appear with any of the following:

'{"filter":{"exists":{"field":"foo"}}}'
'{"filter":{"exists":{"field":"_all"}}}'

Correct. Because the {bar:baz} value for foo is ignored, the 'foo'
field is "missing", and similarly, no values have been passed to '_all',
so that is also "missing".

But it appears on this:

'{"filter":{"script":{"script":"_source.foo.bar == "baz""}}}'

The _source field just returns whatever JSON doc you passed to
elasticsearch. It has nothing to do with what is indexed.

So what I understand up to this point is that the document is stored,
but not indexed. That's what you meant, Clint?

Correct

That said, it seems to be fine if I throw in an integer there instead
of a string:

curl -XPUT localhost:9200/test/test/111 -d '{"foo":2}'

curl -XPOST localhost:9200/test/test/_search?pretty=true -d '{

"query": { "match": { "foo":2}}}'

curl -XPOST localhost:9200/test/test/_search?pretty=true -d

'{"filter":{"range":{"foo":{"from": 1, "to": 3}}}}'

Yes, value 2 becomes "2" because it is type 'string'. Note, however,
that your range is string based, not number based.

I was suspecting that. Thanks for confirming :slight_smile:

Best regards,
Radu

http://sematext.com/ -- ElasticSearch -- Solr -- Lucene

--


(govind201-2) #15

Ah, so it's "stored" by not "indexed". Thanks for clearing that up. Guess I
will have to reindex.
Thanks for your help all.

On Thursday, October 25, 2012 5:53:16 PM UTC+8, Radu Gheorghe wrote:

Hi Clinton,

On Thu, Oct 25, 2012 at 12:26 PM, Clinton Gormley <cl...@traveljury.com<javascript:>>
wrote:

Hi Radu

So yesterday I tested before writing here what's described below and
it worked. I just copy-pasted here. Now, I can't reproduce the same
behavior (went through the history - I still don't get it). Instead, I
get what govind201 reported. I was using 0.19.11 all this time.

I think you probably deleted the index then indexed the doc, which
created a mapping for the 'foo' field of type 'object'.

Neah, I checked through the history and I didn't do that. Plus, it
couldn't have ended up as "foo": "bar baz" if I did that, right?

Furthermore, the document "_source" : {"foo":{"bar":"baz"}} doesn't
appear with any of the following:

'{"filter":{"exists":{"field":"foo"}}}'
'{"filter":{"exists":{"field":"_all"}}}'

Correct. Because the {bar:baz} value for foo is ignored, the 'foo'
field is "missing", and similarly, no values have been passed to '_all',
so that is also "missing".

But it appears on this:

'{"filter":{"script":{"script":"_source.foo.bar == "baz""}}}'

The _source field just returns whatever JSON doc you passed to
elasticsearch. It has nothing to do with what is indexed.

So what I understand up to this point is that the document is stored,
but not indexed. That's what you meant, Clint?

Correct

That said, it seems to be fine if I throw in an integer there instead
of a string:

curl -XPUT localhost:9200/test/test/111 -d '{"foo":2}'

curl -XPOST localhost:9200/test/test/_search?pretty=true -d '{

"query": { "match": { "foo":2}}}'

curl -XPOST localhost:9200/test/test/_search?pretty=true -d

'{"filter":{"range":{"foo":{"from": 1, "to": 3}}}}'

Yes, value 2 becomes "2" because it is type 'string'. Note, however,
that your range is string based, not number based.

I was suspecting that. Thanks for confirming :slight_smile:

Best regards,
Radu

http://sematext.com/ -- ElasticSearch -- Solr -- Lucene

--


(system) #16