Problem with BIG_INTEGERs

Elastic Stack Elasticsearch
1

Our application creates a 64-bit hash id that is stored in an
ElastichSearch long field. We have some problems retrieving
documents. Retrieving "_source" works. Trying to retrieve any
specific field(s) causes the following error from some shards:

failures: [

{
    status: 500
    reason: RemoteTransportException[[dev-nosql-h03][inet[/

172.22.59.53:9300]][search/phase/fetch/id]]; nested:
ElasticSearchParseException[failed to parse / load source]; nested:
ElasticSearchIllegalStateException[No matching token for number_type
[BIG_INTEGER]];
}

Here are some example integers that cause the issue:

0xB8F136795037CFD0
0x9BE37E6093517F88

Our application only needs 48bits hash ID's. I reindex everything
with 48bit values and the problem went away.

ElasticSearch 0.18.6. The index stores _source. It does not store
any individual fields.

2

I assume it is a problem reading an unsigned value into a signed
variable.

3

Can you create a recreation? Even when indexing, if it works, your numbers
will loose resolution if they are too big since they are converted to a
long number.

On Mon, Jan 9, 2012 at 9:49 PM, drew.letcher <
drew.letcher@interactivedata.com> wrote:

I assume it is a problem reading an unsigned value into a signed
variable.

4

I'm a C++ programmer using the HTTP interface. I didn't realize that
Java doesn't have unsigned integers.

I understand what you are saying. Java is not able to parse the value
so that field would not be indexed. I can see them in the _source,
because that is the exact json text I sent. A search asking for
specifc fields causes the _source to be parsed which generates the
Java error.

If I wanted 64 bit hash ID's I would have to cast the unsigned value
to a signed value before passing it to ElasticSearch.

Mystery solved.

5

Actually I have the same problem when running ES 0.19.3. I also have 64-bit
hashed id for many different attributes in our documents, but it seems ES
can't automatically recognize these id's:

dxu@ubuntu:~$ curl -s -S -XPUT 'http://localhost:9200/test/test1/1' -d
'{"id": 11516060740533628129}'
{"error":"ElasticSearchIllegalStateException[No matching token for
number_type [BIG_INTEGER]]","status":500}

On Monday, January 9, 2012 2:58:29 PM UTC-5, kimchy wrote:

Can you create a recreation? Even when indexing, if it works, your numbers
will loose resolution if they are too big since they are converted to a
long number.

On Mon, Jan 9, 2012 at 9:49 PM, drew.letcher <
drew.letcher@interactivedata.com> wrote:

I assume it is a problem reading an unsigned value into a signed
variable.

6

Thats because ES supporting 64bit signed integers, just send it as a string
in this case.

On Sat, May 26, 2012 at 1:24 AM, calc donghua@gmail.com wrote:

Actually I have the same problem when running ES 0.19.3. I also have
64-bit hashed id for many different attributes in our documents, but it
seems ES can't automatically recognize these id's:

dxu@ubuntu:~$ curl -s -S -XPUT 'http://localhost:9200/test/test1/1' -d
'{"id": 11516060740533628129}'
{"error":"ElasticSearchIllegalStateException[No matching token for
number_type [BIG_INTEGER]]","status":500}

On Monday, January 9, 2012 2:58:29 PM UTC-5, kimchy wrote:

Can you create a recreation? Even when indexing, if it works, your
numbers will loose resolution if they are too big since they are converted
to a long number.

On Mon, Jan 9, 2012 at 9:49 PM, drew.letcher <
drew.letcher@interactivedata.**com drew.letcher@interactivedata.com>wrote:

I assume it is a problem reading an unsigned value into a signed
variable.

1 Like