Unable to index into elasticsearch due to Byte range being out of range

[Shreesh_Narayanan]

Hi,

I have a log that shows the interface usage (eth0/eth1) at a particular time , it logs in bytes and while logstash is able to parse it , elasticsearch seems to be rejecting it . Any workaround for this ? , in the component template , i gave the mapping as. "bytes_written" -> "numeric/byte"

here's a sample log

1 1678945972 <REDACTED> eth0 <REDACTED> 36248274 37294982749 56653568 82227069534

Here's what the sample logstash output shows

[2023-03-16T13:04:54,826][WARN ][logstash.outputs.elasticsearch][interface_logger_log][8acbe052951cec868f387d4444fb955dc087a4bb4c48c6852ef35b8cee41737d] Could not index event to Elasticsearch. status: 400, action: ["create", {:_id=>nil, :_index=>interfacelogger_lines", :routing=>nil}, {"num_packets_written"=>"3230996", "bytes_written"=>"3577587849", "interface_logger_version"=>"1", "@timestamp"=>2023-03-16T09:45:34.000Z, "bytes_read"=>"632120356", "interface_ip"=>"$REDACTED", "num_packets_read"=>"2566696", "interface"=>"eth0"}], response: {"create"=>{"_index"=>"logstash_interface_logger_lines-000001", "_id"=>"gHmF6oYBVK7ZTxZd_N8N", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [bytes_written] of type [byte] in document with id 'gHmF6oYBVK7ZTxZd_N8N'. Preview of field's value: '3577587849'", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Value [3577587849] is out of range for an integer"}}}}

[leandrojmp (Leandro Pereira)]

This is a mapping issue, the field is mapped as a byte but the value is higher than what an byte field would support, this should be mapped as long.

@Shreesh_Narayanan

in the component template , i gave the mapping as. "bytes_written" -> "numeric/byte"

The byte here is the byte data type, it only supports values between -128 and 127, this is your issue, the field should be mapped as long, you can check the numeric data types in this documentation.

The solution is to change the mapping and recreate the index.

[Shreesh_Narayanan]

Okay , wil try this and report back. Thanks a bunch

[Shreesh_Narayanan]

this worked