I have an index template which defines the data type for the field liableban
as follows:
"liableBan": {
"meta": {
"entity": "usageCharacteristic.liableBan"
},
"null_value": 0,
"store": true,
"type": "long"
}
If I check the value of this field in elastic, it is set as follows:
"liableBan": [
-7193973785499238000
],
If I query using curl
, as follows:
response=$(curl -s --insecure -u "$ELASTIC_USER:$ELASTIC_PASSWORD" -X GET "$ELASTIC_URL/$INDEX/_search" -H 'Content-Type: application/json' -d '{
"query": {
"term": {
"eventId": "'$EVENT_ID'"
}
}
}')
echo $response
It comes out as follows:
"liableBan":-7193973785499238559
The last three digits are different. If I pipe the result into jq
, I get the correct value back:
"liableBan": -7193973785499238000,
I am getting the same issue with the python elastic library where it is coming out as ...8559 instead of 8000
Seems like it, like in the issue discussed in this thread , could be a limitation around the numeric precision supported by JSON.
2 Likes
Thanks @Christian_Dahlqvist
The only workaround I could find in python was to just pass the response to jq. I tried many other suggestions from chatgpt
def call_jq(json_data):
if isinstance(json_data, dict):
json_data = json.dumps(json_data)
try:
result = subprocess.run(
["jq", "."],
input=json_data,
text=True,
capture_output=True,
check=True
)
return json.loads(result.stdout.strip())
except subprocess.CalledProcessError as e:
print(f"Error: {e.stderr}")
return None
just call that method after getting the response back:
response = es.search(index="_all", body=query)
if response['hits']['hits']:
es_data = response['hits']['hits'][0]['_source']
fixed_data = call_jq(es_data)
return clean_data(fixed_data)
else:
return None
RainTown
(Kevin Maguire)
January 15, 2025, 11:55pm
4
Hi
Wild idea - just don’t store a floating point number of that absolute size into a float field. That’s a form of hidden technical debt you’re gifting to others down the line. Look to re-engineer a bit. I’ve no idea what liableBan corresponds to in real world sense, but I suspect you or your team might.
Patient: Doctor doctor, it hurts when I do this?
Doctor: Don’t do it then!