Strange problem with a boolean field

Ian_Clark1 · December 11, 2012, 1:35am

Hi,

has anyone had a problem with a boolean field having a value 't' when
normally it is 'T' or 'F' (for true and false)? It's happens to me
occasionally in my integration tests. I've tried to simplify the problem
into a gist, but annoyingly it doesn't happen when I do that... grr.

A screenshot of the problem:-

https://docs.google.com/open?id=0B3kE2Ho2wIf5ZjltNjJKNEJ2cG8

here's roughly what happens as a gist:-

gist.github.com

https://gist.github.com/ianAndrewClark/98a3287dea857ec12ec9

argh-test.sh


curl -X DELETE "http://localhost:9200/iotest"
echo -e "\ndeleted index"

curl -XPUT 'http://localhost:9200/iotest/' -d '
{
  "settings": {
    "index": {
      "number_of_shards": 5,
      "number_of_replicas": 0

This file has been truncated. show original

(but that actually works for me)

Any ideas?

--

Kay_Ropke · December 11, 2012, 10:15am

Hi!

On 11.12.2012, at 02:35, Ian Clark wrote:

has anyone had a problem with a boolean field having a value 't' when normally it is 'T' or 'F' (for true and false)? It's happens to me occasionally in my integration tests. I've tried to simplify the problem into a gist, but annoyingly it doesn't happen when I do that... grr.

You've come across this: Facet value of boolean mapped fields should be "true" or "false" · Issue #2462 · elastic/elasticsearch · GitHub
Boolean values are stored as 'T' and 'F' in the index, but aren't properly converted back to 'true' and 'false' when being returned to a client.
There's a workaround and a patch, but I guess it will take a little while to get it in a release.

cheers,
-k

--

Ian_Clark1 · December 11, 2012, 11:55am

Where does my little 't' value come from? It is a problem for me in that:-
I then filter on true, and get back all results minus the little 't'
document:-

like so:- failing-filter.json · GitHub

IC

On Tue, Dec 11, 2012 at 10:15 AM, Kay Röpke kroepke@gmail.com wrote:

Hi!

On 11.12.2012, at 02:35, Ian Clark wrote:

has anyone had a problem with a boolean field having a value 't' when
normally it is 'T' or 'F' (for true and false)? It's happens to me
occasionally in my integration tests. I've tried to simplify the problem
into a gist, but annoyingly it doesn't happen when I do that... grr.

You've come across this:
Facet value of boolean mapped fields should be "true" or "false" · Issue #2462 · elastic/elasticsearch · GitHub
Boolean values are stored as 'T' and 'F' in the index, but aren't properly
converted back to 'true' and 'false' when being returned to a client.
There's a workaround and a patch, but I guess it will take a little while
to get it in a release.

cheers,
-k

--

--

Igor_Motov · December 13, 2012, 9:32pm

I tried it with 0.20.1 and I cannot reproduce it. Which version of es and
java are you using? Is it reproducible on a fresh install of elasticsearch?
All I am getting is

"facets" : {
"verified_terms" : {
"_type" : "terms",
"missing" : 0,
"total" : 30,
"other" : 0,
"terms" : [ {
"term" : "T",
"count" : 30
} ]
}
}

On Tuesday, December 11, 2012 6:55:54 AM UTC-5, Ian Clark wrote:

Where does my little 't' value come from? It is a problem for me in that:-
I then filter on true, and get back all results minus the little 't'
document:-

like so:- failing-filter.json · GitHub

IC

On Tue, Dec 11, 2012 at 10:15 AM, Kay Röpke <kro...@gmail.com<javascript:>

wrote:

Hi!

On 11.12.2012, at 02:35, Ian Clark wrote:

has anyone had a problem with a boolean field having a value 't' when
normally it is 'T' or 'F' (for true and false)? It's happens to me
occasionally in my integration tests. I've tried to simplify the problem
into a gist, but annoyingly it doesn't happen when I do that... grr.

You've come across this:
Facet value of boolean mapped fields should be "true" or "false" · Issue #2462 · elastic/elasticsearch · GitHub
Boolean values are stored as 'T' and 'F' in the index, but aren't
properly converted back to 'true' and 'false' when being returned to a
client.
There's a workaround and a patch, but I guess it will take a little while
to get it in a release.

cheers,
-k

--

--

Ian_Clark1 · December 14, 2012, 12:10pm

Hi,

I was using a snapshot when I ran it manually and 0.20.1 in my integration
test, having sorted that out I can now reproduce the problem with this
simpler script:-

gist.github.com

https://gist.github.com/ianAndrewClark/4284891

boolean-flag-problem.sh

echo "\nSetup test index"
curl -XDELETE "http://localhost:9200/booltest"
curl -XPUT 'http://localhost:9200/booltest/' -d '
{
  "settings": {
    "index": {
      "number_of_shards": 1,
      "number_of_replicas": 0
    },
    "mappings": {

This file has been truncated. show original

sh output

Setup test index
{"ok":true,"acknowledged":true}{"ok":true,"acknowledged":true}
ok to start
{"cluster_name":"elasticsearch","status":"green","timed_out":false,"number_of_nodes":1,"number_of_data_nodes":1,"active_primary_shards":2,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0}
doc 1
{"ok":true,"_index":"booltest","_type":"thing","_id":"1","_version":1}
doc 2
{"ok":true,"_index":"booltest","_type":"thing","_id":"2","_version":1}
doc 3
{"ok":true,"_index":"booltest","_type":"thing","_id":"3","_version":1}

This file has been truncated. show original

My java version is 1.7u9. Do this fail for you?

thanks for looking.

On Thu, Dec 13, 2012 at 9:32 PM, Igor Motov imotov@gmail.com wrote:

I tried it with 0.20.1 and I cannot reproduce it. Which version of es and
java are you using? Is it reproducible on a fresh install of elasticsearch?
All I am getting is

"facets" : {
"verified_terms" : {
"_type" : "terms",
"missing" : 0,
"total" : 30,
"other" : 0,
"terms" : [ {
"term" : "T",
"count" : 30
} ]
}
}

On Tuesday, December 11, 2012 6:55:54 AM UTC-5, Ian Clark wrote:

Where does my little 't' value come from? It is a problem for me in
that:- I then filter on true, and get back all results minus the little 't'
document:-

like so:- https://gist.github.com/**26a7f43bcbbfbff836a6 https://gist.github.com/26a7f43bcbbfbff836a6

IC

On Tue, Dec 11, 2012 at 10:15 AM, Kay Röpke kro...@gmail.com wrote:

Hi!

On 11.12.2012, at 02:35, Ian Clark wrote:

has anyone had a problem with a boolean field having a value 't' when
normally it is 'T' or 'F' (for true and false)? It's happens to me
occasionally in my integration tests. I've tried to simplify the problem
into a gist, but annoyingly it doesn't happen when I do that... grr.

You've come across this: https://github.com/**
elasticsearch/elasticsearch/**issues/2462https://github.com/elasticsearch/elasticsearch/issues/2462
Boolean values are stored as 'T' and 'F' in the index, but aren't
properly converted back to 'true' and 'false' when being returned to a
client.
There's a workaround and a patch, but I guess it will take a little
while to get it in a release.

cheers,
-k

--

--

--

Igor_Motov · December 14, 2012, 5:26pm

Ian,

Thank you for reporting the bug. We are looking into it. As a work around,
you can fix your mapping by moving the "mappings" element to the same level
as "settings":

curl -XPUT 'http://localhost:9200/booltest/' -d '
{
"settings": {
"index": {
"number_of_shards": 1,
"number_of_replicas": 0
}
},
"mappings": {
"thing": {
"properties": {
"flag": { "type" : "boolean" }
},
"_all": {
"enabled": "false"
}
}
}
}
'

Igor

On Friday, December 14, 2012 4:10:21 AM UTC-8, Ian Clark wrote:

Hi,

I was using a snapshot when I ran it manually and 0.20.1 in my integration
test, having sorted that out I can now reproduce the problem with this
simpler script:-

problem with boolean fields · GitHub

My java version is 1.7u9. Do this fail for you?

thanks for looking.

On Thu, Dec 13, 2012 at 9:32 PM, Igor Motov <imo...@gmail.com<javascript:>

wrote:

I tried it with 0.20.1 and I cannot reproduce it. Which version of es and
java are you using? Is it reproducible on a fresh install of elasticsearch?
All I am getting is

"facets" : {
"verified_terms" : {
"_type" : "terms",
"missing" : 0,
"total" : 30,
"other" : 0,
"terms" : [ {
"term" : "T",
"count" : 30
} ]
}
}

On Tuesday, December 11, 2012 6:55:54 AM UTC-5, Ian Clark wrote:

Where does my little 't' value come from? It is a problem for me in
that:- I then filter on true, and get back all results minus the little 't'
document:-

like so:- https://gist.github.com/**26a7f43bcbbfbff836a6 https://gist.github.com/26a7f43bcbbfbff836a6

IC

On Tue, Dec 11, 2012 at 10:15 AM, Kay Röpke kro...@gmail.com wrote:

Hi!

On 11.12.2012, at 02:35, Ian Clark wrote:

has anyone had a problem with a boolean field having a value 't' when
normally it is 'T' or 'F' (for true and false)? It's happens to me
occasionally in my integration tests. I've tried to simplify the problem
into a gist, but annoyingly it doesn't happen when I do that... grr.

You've come across this: https://github.com/**
elasticsearch/elasticsearch/**issues/2462https://github.com/elasticsearch/elasticsearch/issues/2462
Boolean values are stored as 'T' and 'F' in the index, but aren't
properly converted back to 'true' and 'false' when being returned to a
client.
There's a workaround and a patch, but I guess it will take a little
while to get it in a release.

cheers,
-k

--

--

--

Ian_Clark1 · December 14, 2012, 5:55pm

Igor,

I totally missed that, so it was falling back to a dynamic mapping and that
is the problem?

Thanks for looking and the work around.

Ian

On Fri, Dec 14, 2012 at 5:26 PM, Igor Motov imotov@gmail.com wrote:

Ian,

Thank you for reporting the bug. We are looking into it. As a work around,
you can fix your mapping by moving the "mappings" element to the same level
as "settings":

curl -XPUT 'http://localhost:9200/booltest/' -d '
{
"settings": {
"index": {
"number_of_shards": 1,
"number_of_replicas": 0
}
},
"mappings": {
"thing": {
"properties": {
"flag": { "type" : "boolean" }
},
"_all": {
"enabled": "false"
}
}
}
}
'

Igor

On Friday, December 14, 2012 4:10:21 AM UTC-8, Ian Clark wrote:

Hi,

I was using a snapshot when I ran it manually and 0.20.1 in my
integration test, having sorted that out I can now reproduce the problem
with this simpler script:-

https://gist.github.com/**4284891 https://gist.github.com/4284891

My java version is 1.7u9. Do this fail for you?

thanks for looking.

On Thu, Dec 13, 2012 at 9:32 PM, Igor Motov imo...@gmail.com wrote:

I tried it with 0.20.1 and I cannot reproduce it. Which version of es
and java are you using? Is it reproducible on a fresh install of
elasticsearch? All I am getting is

"facets" : {
"verified_terms" : {
"_type" : "terms",
"missing" : 0,
"total" : 30,
"other" : 0,
"terms" : [ {
"term" : "T",
"count" : 30
} ]
}
}

On Tuesday, December 11, 2012 6:55:54 AM UTC-5, Ian Clark wrote:

Where does my little 't' value come from? It is a problem for me in
that:- I then filter on true, and get back all results minus the little 't'
document:-

like so:- https://gist.github.com/**2**6a7f43bcbbfbff836a6 https://gist.github.com/26a7f43bcbbfbff836a6

IC

On Tue, Dec 11, 2012 at 10:15 AM, Kay Röpke kro...@gmail.com wrote:

Hi!

On 11.12.2012, at 02:35, Ian Clark wrote:

has anyone had a problem with a boolean field having a value 't' when
normally it is 'T' or 'F' (for true and false)? It's happens to me
occasionally in my integration tests. I've tried to simplify the problem
into a gist, but annoyingly it doesn't happen when I do that... grr.

You've come across this: https://github.com/**elast**
icsearch/elasticsearch/**issues/**2462https://github.com/elasticsearch/elasticsearch/issues/2462
Boolean values are stored as 'T' and 'F' in the index, but aren't
properly converted back to 'true' and 'false' when being returned to a
client.
There's a workaround and a patch, but I guess it will take a little
while to get it in a release.

cheers,
-k

--

--

--

--

Igor_Motov · December 14, 2012, 5:58pm

Yes, in your example it's effectively using dynamic mapping.

On Friday, December 14, 2012 9:55:56 AM UTC-8, Ian Clark wrote:

Igor,

I totally missed that, so it was falling back to a dynamic mapping and
that is the problem?

Thanks for looking and the work around.

Ian

On Fri, Dec 14, 2012 at 5:26 PM, Igor Motov <imo...@gmail.com<javascript:>

wrote:

Ian,

Thank you for reporting the bug. We are looking into it. As a work
around, you can fix your mapping by moving the "mappings" element to the
same level as "settings":

curl -XPUT 'http://localhost:9200/booltest/' -d '
{
"settings": {
"index": {
"number_of_shards": 1,
"number_of_replicas": 0
}
},
"mappings": {
"thing": {
"properties": {
"flag": { "type" : "boolean" }
},
"_all": {
"enabled": "false"
}
}
}
}
'

Igor

On Friday, December 14, 2012 4:10:21 AM UTC-8, Ian Clark wrote:

Hi,

I was using a snapshot when I ran it manually and 0.20.1 in my
integration test, having sorted that out I can now reproduce the problem
with this simpler script:-

https://gist.github.com/**4284891 https://gist.github.com/4284891

My java version is 1.7u9. Do this fail for you?

thanks for looking.

On Thu, Dec 13, 2012 at 9:32 PM, Igor Motov imo...@gmail.com wrote:

I tried it with 0.20.1 and I cannot reproduce it. Which version of es
and java are you using? Is it reproducible on a fresh install of
elasticsearch? All I am getting is

"facets" : {
"verified_terms" : {
"_type" : "terms",
"missing" : 0,
"total" : 30,
"other" : 0,
"terms" : [ {
"term" : "T",
"count" : 30
} ]
}
}

On Tuesday, December 11, 2012 6:55:54 AM UTC-5, Ian Clark wrote:

Where does my little 't' value come from? It is a problem for me in
that:- I then filter on true, and get back all results minus the little 't'
document:-

like so:- https://gist.github.com/**2**6a7f43bcbbfbff836a6 https://gist.github.com/26a7f43bcbbfbff836a6

IC

On Tue, Dec 11, 2012 at 10:15 AM, Kay Röpke kro...@gmail.com wrote:

Hi!

On 11.12.2012, at 02:35, Ian Clark wrote:

has anyone had a problem with a boolean field having a value 't' when
normally it is 'T' or 'F' (for true and false)? It's happens to me
occasionally in my integration tests. I've tried to simplify the problem
into a gist, but annoyingly it doesn't happen when I do that... grr.

You've come across this: https://github.com/**elast**
icsearch/elasticsearch/**issues/**2462https://github.com/elasticsearch/elasticsearch/issues/2462
Boolean values are stored as 'T' and 'F' in the index, but aren't
properly converted back to 'true' and 'false' when being returned to a
client.
There's a workaround and a patch, but I guess it will take a little
while to get it in a release.

cheers,
-k

--

--

--

--