Java REST client returning long value in Exponential

Elastic Stack Elasticsearch
1

Hi,

I am using java REST client of Elasticsearch. I have a requirement where i need to find the max value of a long field from all the documents in an Index.

I ran a POST http://localhost:9200/myIndex/myType/_search?size=0

with body
{
"aggs" : {{
"took": 2,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 995,
"max_score": 0,
"hits": []
},
"aggregations": {
"MYID": {
"value": 1805130000005357060
}
}
}
"MYID" : { "max" : { "field" : "MYID" } }
}
}

I get the output,

"aggregations" : {
"MYID" : {
"value" : 1.80513000000535706E18
}
}

How can I work around this? For now I worked around this using BigDecimal in my java code. Is there any smarter way to do it?

2

How do you get that output. Can you share the code you are using?

3

Hi @cbuescher

I am using the following Java code to get the data.

HttpEntity httpEntity = new NStringEntity(
					requestJSON.toString(),
					ContentType.APPLICATION_JSON);
			response = restClient.performRequest(Constants.HTTP_METHOD_GET, 
					queryString.toString(), Collections.singletonMap(
								FieldNameConstant.PRETTY,
				        		Boolean.TRUE.toString()), httpEntity);
			String responseBody = EntityUtils.toString(response.getEntity());
			logger.info("response array ["+responseBody+"]");
			JSONObject jsonResp = new JSONObject(responseBody);
			JSONObject jsonAgg = (JSONObject)jsonResp.get(FieldNameConstant.ES_AGGREGATIONS);
			JSONObject jsonValue = (JSONObject)jsonAgg.get(idFieldName);
			BigDecimal bigValue = jsonValue.getBigDecimal(FieldNameConstant.ES_VALUE);

I tried using big decimal, big integer and long. I am loosing precision in every case.

Also, after I create a new document with the next number, when I run the max id query after that, I get the old max value again.

ie. I run the max query in my first post I get 1805130000005357060. Now I create a new document with MYid as 1805130000005357061. Now when I run the max query again, I get 1805130000005357060. This is perplexing, not sure if this has anything to do with precision loss. This is happening when I test with POSTMAN also, So i guess it may not be an issue with the REST Client. But now the issue is how do I find the max value of the long field from all my documents?

Thanks
Paddy

4

@Paddy_Mahadeva sorry for the long delay, your original analysis that this is not only an issue with the REST client is right. The problem is the max-aggregation: all aggregations internally use a double representation because it makes sense for things like sums etc... but for min/max and for very large values this can come at the cost of precision. Here is an issue that discusses this problem.

5

One possible workaround could be to sort on the MYID field and retrieve only the topmost document:

POST http://localhost:9200/test/t/_search
{
  "size": 1, 
  "sort": [
    {
      "MYID": {
        "order": "desc"
      }
    }
  ]
}

{
	[...]
	"hits": {
		"total": 1,
		"max_score": null,
		"hits": [
			{
				"_index": "test",
				"_type": "t",
				"_id": "1",
				"_score": null,
				"_source": {
					"MYID": 1805130000005357061
				},
				"sort": [
					1805130000005357061
				]
			}
		]
	}
}

Does this work for your use case?

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.