Boolean field


(sidpao) #1

When updating a field with a script, ES allows me to set a field mapped as
boolean with any string.
So far, this seems to be the only scenario where is possible to update a
field with a different type than the one defined in the mapping.
Anyone encountered this issue before?

Thanks in advance

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ef269806-3fe8-4f52-8d10-f68e849b466a%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Binh Ly-2) #2

Yup, 0 or false is false. Everything else is true. I do agree it is "odd".

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/59cb17be-a99e-4719-b8bd-ef3e53b212bb%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Brian Yoder) #3

But be careful: the docs for the Java 7 Boolean.parseBooleanhttp://docs.oracle.com/javase/7/docs/api/java/lang/Boolean.html#parseBoolean(java.lang.String)say that:

The boolean returned represents the value true if the string argument is
not null and is equal, ignoring case, to the string"true".

And by implication, everything else is false.

Brian

On Friday, February 28, 2014 8:32:11 AM UTC-5, Binh Ly wrote:

Yup, 0 or false is false. Everything else is true. I do agree it is "odd".

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0dca2278-1c4f-4d72-ab90-88fcc7bc697f%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(sidpao) #4

Good point. So Java's logic is completely opposite to ES. That's going to
make life a bit difficult when using Java client.
Anyone knows if this is intended behaviour or a oversight?
In my case, it's making my updates extremely difficult. I don't know, in
the moment of the update, the type of field being updated. So, i don't know
if the field being updated is a boolean or not. As of now, the only
solution i'm able to think of is to query for the index/type mappings,
check which fields are mapped as boolean, store those fields and
cross-match them with the fields being updated... For any other fields, i
rely on the exceptions thrown by ES for type mismatch.

On Friday, February 28, 2014 2:37:03 PM UTC, InquiringMind wrote:

But be careful: the docs for the Java 7 Boolean.parseBooleanhttp://docs.oracle.com/javase/7/docs/api/java/lang/Boolean.html#parseBoolean(java.lang.String)say that:

The boolean returned represents the value true if the string argument is
not null and is equal, ignoring case, to the string"true".

And by implication, everything else is false.

Brian

On Friday, February 28, 2014 8:32:11 AM UTC-5, Binh Ly wrote:

Yup, 0 or false is false. Everything else is true. I do agree it is "odd".

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/833d9c93-ebcd-4bb4-bf7d-99b094f30eb6%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Binh Ly-2) #5

Brian, the interesting part is the ES boolean is actually a T/F field and
is parsed internally in a different way (not like the Java boolean).

I just looked in the code and the values "", "0", "false", "off", "no" are
all false. Everything else is true. :slight_smile:

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8a51a641-6666-4619-bfd4-2f20f44cdb4c%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Brian Yoder) #6

Binh,

It's even more interesting. The behavior you describe matches the
configuration and REST parsing rules. But the document _source is JSON and
a boolean value must be either true or false. So my recommendation is
to use the configuration and REST interface as a forgiving practice to get
used to specifying only true or false for boolean parameters, and then
you'll be comfortably set when you start adding real data.

And it's one more reason to lock down Elasticsearch to prevent it from
automatically detecting field types, and then explicitly map all of the
fields you need.

For example, the pretty= HTTP REST parameter is a boolean. And here are the
results:

brian@liberator:~$ curl -XGET 'http://localhost:9200/_nodes/process?
pretty=false&human=true' && echo
{"cluster_name":"brian-liberator","nodes":{"wY-tmDrATweDeltdz9xCiQ":{"name":"liberator","transport_address":"inet[/192.168.1.70:9300]","host":"liberator","ip":"127.0.1.1","version":"1.0.0","build":"a46900e","http_address":"inet[/192.168.1.70:9200]","attributes":{"max_local_storage_nodes":"1"},"process":{"refresh_interval":1000,"id":2864,"max_file_descriptors":65537,"mlockall":false}}}}

brian@liberator:~$ curl -XGET 'http://localhost:9200/_nodes/process?
pretty=true&human=true' && echo
{
"cluster_name" : "brian-liberator",
"nodes" : {
"wY-tmDrATweDeltdz9xCiQ" : {
"name" : "liberator",
"transport_address" : "inet[/192.168.1.70:9300]",
"host" : "liberator",
"ip" : "127.0.1.1",
"version" : "1.0.0",
"build" : "a46900e",
"http_address" : "inet[/192.168.1.70:9200]",
"attributes" : {
"max_local_storage_nodes" : "1"
},
"process" : {
"refresh_interval" : 1000,
"id" : 2864,
"max_file_descriptors" : 65537,
"mlockall" : false
}
}
}
}

brian@liberator:~$ curl -XGET 'http://localhost:9200/_nodes/process?
pretty=AsPrettyDoes&human=true' && echo
{
"cluster_name" : "brian-liberator",
"nodes" : {
"wY-tmDrATweDeltdz9xCiQ" : {
"name" : "liberator",
"transport_address" : "inet[/192.168.1.70:9300]",
"host" : "liberator",
"ip" : "127.0.1.1",
"version" : "1.0.0",
"build" : "a46900e",
"http_address" : "inet[/192.168.1.70:9200]",
"attributes" : {
"max_local_storage_nodes" : "1"
},
"process" : {
"refresh_interval" : 1000,
"id" : 2864,
"max_file_descriptors" : 65537,
"mlockall" : false
}
}
}
}

But JSON is not so forgiving. Here is a JSON document, followed by the
output from a parser application I wrote last year:

brian@liberator:~$ cat /tmp/test.json
{
"v1" : true,
"v2" : false,
"v3" : 0,
"v4" : "0",
"v5" : 1,
"v6" : "1",
"v7" : TRUE
}

brian@liberator:~$ parse-json.sh /tmp/test.json
JSON PARSED VALUES:
((NAME null)(TEXT {') (TOKEN START_OBJECT)) ((NAME v1)(TEXTtrue') (TOKEN VALUE_TRUE))
((NAME v2)(TEXT false') (TOKEN VALUE_FALSE)) ((NAME v3)(TEXT0') (TOKEN VALUE_NUMBER_INT))
((NAME v4)(TEXT 0') (TOKEN VALUE_STRING)) ((NAME v5)(TEXT1') (TOKEN VALUE_NUMBER_INT))
((NAME v6)(TEXT `1') (TOKEN VALUE_STRING))
*JSON parse failure:
org.elasticsearch.common.jackson.core.JsonParseException: Unrecognized
token 'TRUE': was expecting *

  • at [Source: java.io.BufferedReader@1662dc8; line: 8, column: 14]*

Brian

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/357cbf17-fab4-428f-a88d-d50cdd49110f%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(sidpao) #7

Regarding the mapping, i have it manually defined as boolean. I too don't like auto mapping. And yet, ES accepts any string and, when i perform a get, also returns any string instead of true/false (doesn't converte before upgrading ir fetching). Even funnier is that it accepts a real boolean (true/false without quotes) and a true/false with quotes, never throwing exceptions.
Also tested it in a "doc" section instead of script, same results. It seems to fully ignore the mapping and assumes it as a string.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8fb0d4a6-cf24-470b-b299-8704f5e2f5a2%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(sidpao) #8

So far this issue seems unavoidable. Did some more testing / googling and
did not find any suitable solution.
I believe the correct behaviour here (if sending anything other than true,
false or null) would be to throw a ElasticsearchException of some sorts.

In the end, i circled the issue by defining a helper method that, for every
key/value pair that's about to be updated checks it's mapping. The method
simply receives an index name, type and field and returns either null or
the type defined for that field.
Back in my update method, i call that helper method for every field being
updated. If any of the types defined as boolean receives a value different
than true, false or null, i throw a custom-made exception. For the rest of
them, i let ES do the check for me (so far the error throw for incorrect
field type has been a script error)
In my case, given some of my requirements (1. Can't update a field that
doesn't exists in the mapping and 2. Types must be enforced), i couldn't
think of any other way of circumventing the issue. If anyone finds a better
solution, please let me know.

Thanks for all the help

On Friday, February 28, 2014 12:44:54 PM UTC, sid...@gmail.com wrote:

When updating a field with a script, ES allows me to set a field mapped as
boolean with any string.
So far, this seems to be the only scenario where is possible to update a
field with a different type than the one defined in the mapping.
Anyone encountered this issue before?

Thanks in advance

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/3b1250b5-b7ca-420b-9a0c-600e2e812974%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #9