Fix field values while importing

Elastic Stack Logstash
1

Hey,

When I fetch events from S3 into ES i'm getting multiple errors like this:

Caused by: org.elasticsearch.common.jackson.core.JsonParseException: Numeric value (9223372036854776000) out of range of long (-9223372036854775808 - 9223372036854775807)

This error is show on ES. The error on LS is something like:

"... failed action with response of 400, dropping action ... "

I understand that some of my events have a numeric value that exceeds the maximum allowed by ES. Is it possible that whenever LS sees then number '9223372036854776000' it'll convert it to '9223372036854775807' ?

2

My only thought is using the ruby filter

Of course if you keep the value as a String you will not have this issue :smile: but you probably want to use it for charting

filter{
if [ bigint ]{
ruby {
    code => "event['duration_down'] = event['hour']*3600 + event['min']*60 + event['sec']"
}
}
}
3

Good idea. I'll try that.