[bug] big decimal value losed precision

Elastic Stack Elasticsearch
1

Hello, ES team!
There is a precision losing problem when I'm using java client of ElasticSearch.
Will you fix this bug recently? We do need a big decimal in developing financial system.
Thanks very much!

[Development Environment]
JDK: 1.8
ElasticSearch Server Version: 6.4.2 / 6.8.9 / 7.9.0
ElasticSearch Client Version: 6.4.2 / 6.8.9 / 7.9.0

[Mapping]
PUT test_idx/_doc/_mapping
{
"properties": {
"balance":{"type": "double"}
}
}

[Main Codes & Results -- version of java client and ES server: 6.4.2 / 6.8.9]
BigDecimal val = new BigDecimal("12345678901234567890.12345");
JSONObject jsonObject = new JSONObject();
jsonObject.put("balance", val);

/* Single Put */
// The field 'balance' in curl result by shell showed the correct value '12345678901234567890.12345'
client.index(new IndexRequest("test_idx", "_doc", "1").source(jsonObject.toJSONString(), XContentType.JSON)
        ,RequestOptions.DEFAULT);

/* Bulk Load */
// The field 'balance' in curl result by shell showed the precision losing value '1.2345678901234567E19'
BulkRequest bulkRequest = new BulkRequest();
bulkRequest.add(new IndexRequest("test_idx", "_doc", "2").source(jsonObject.toJSONString(), XContentType.JSON));
client.bulk(bulkRequest, RequestOptions.DEFAULT);

[Main Codes & Results -- version of java client and ES server: 7.9.0]
BigDecimal val = new BigDecimal("12345678901234567890.12345");
JSONObject jsonObject = new JSONObject();
jsonObject.put("balance", val);

/* Single Put */
// The field 'balance' in curl result by shell showed the correct value '12345678901234567890.12345'
client.index(new IndexRequest("test_idx").source(jsonObject.toJSONString(), XContentType.JSON)
        ,RequestOptions.DEFAULT);

/* Bulk Load */
// The field 'balance' in curl result by shell showed the precision losing value '1.2345678901234567E19'
BulkRequest bulkRequest = new BulkRequest();
bulkRequest.add(new IndexRequest("test_idx").source(jsonObject.toJSONString(), XContentType.JSON));
client.bulk(bulkRequest, RequestOptions.DEFAULT);

[Debug and Modify code in java client of ES 7.9.0]
From the results above in different version we can see that it'll losing precision when bulk loading data into ES.
So I debugged code, and I found this bug occured in function '_parseSlowFloat(int expType)' of class 'jackon-core-2.10.4.jar/com.fasterxml.jackson.core.base.ParserBase', the value of variable 'expType' is always 0(int value) from begging, it havn't changged, so the non-int numeric value will be regarded as type 'double' always.
And then, I done these:
[1] downloaded the source code 'jackson-core-2.10.4-sources.jar' and decompressed it into my project.
[2] altered code
/********* before /
if (expType == NR_BIGDECIMAL) {
_numberBigDecimal = _textBuffer.contentsAsDecimal();
_numTypesValid = NR_BIGDECIMAL;
} else {
// Otherwise double has to do
_numberDouble = _textBuffer.contentsAsDouble();
_numTypesValid = NR_DOUBLE;
}
/ after *********/
_numberBigDecimal = _textBuffer.contentsAsDecimal();
_numTypesValid = NR_BIGDECIMAL;
[3] rerun bulk load code to ES, field 'balance' in curl result by shell showed the correct value '12345678901234567890.12345'

2

Hello, is there anybody who would reply to me? :thinking:

3

Hi,

Please be patient with the users in this forum - you opened the topic on friday and today is just monday. If you need faster responses to your questions you might want to order a paid license from Elastic which gives you a guaranteed SLA tickets.

Also, I saw that you already created an issue in Github here. As this seems to be a bug you might get a better answer there as this forum is more about discussions and questions.

Best regards
Wolfram

4

Got it, thanks.
I will be patient to wait for other guys' replys. :blush:

5

If I recall correctly I believe there is a limit to the precision JSON supports.

6

Actually sourcing map data to bulk load ES meets the same result: losing pricesion, but putting one by one record of sourcing map data works fine.

7

I don't think this is a bug but a limitation, please check this issue:

8

Thanks for your reply. But I'm still confused why putting single record with bigdecimal to ES works fine, while bulk loading multis loses precision...It's unfair to them :rofl:

9

Hi, Ignacio_Vera.
I'm still considering it as a bug. Did you see my description above?
When I downloaded the source code 'jackson-core-2.10.4-sources.jar' and decompressed it into my project, and then altered codes in java class 'ParserBase.java' as below:
====================================================================
/********* codes before *********/
if (expType == NR_BIGDECIMAL) {
_numberBigDecimal = _textBuffer.contentsAsDecimal();
_numTypesValid = NR_BIGDECIMAL;
} else {
// Otherwise double has to do
_numberDouble = _textBuffer.contentsAsDouble();
_numTypesValid = NR_DOUBLE;
}

/********* codes after *********/
_numberBigDecimal = _textBuffer.contentsAsDecimal();
_numTypesValid = NR_BIGDECIMAL;
====================================================================

After finished altering codes, I reran codes of putting single big-decimal record and bulk loaded multi big-decimal records, both of them worked fine, big-decimal value stored correctly into ES(should be queryed by curl shell, not in kibana or web browser).

So, don't you think this is a bug?

10

That sounds to me an issue in the Jackson library not in Elasticsearch?

11

I'm not sure it's an issue in the Jackson library, but Elasticsearch stored the precision losing value finally, and it can be revised after altering Jackson library codes. So I used 'bug' in the topic name. :rofl:
I think you can make a try, and then draw a conclusion. Will you?

12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.