Ruby Code to split key value pair in Logstash 6.8.0

Elastic Stack Logstash
1 
json
{
source => "message"
}
split {
field => "data"
	}
date {
  match => ["[data][currentDate]", "yyyy-MM-dd HH:mm:ss"]
  target => "@timestamp"
}
ruby {
          code => "
          event['data'].each {|k, v|
          event[k] = v
          }
          event.remove('data')
        "
      }

This was used earlier in logstash 1.5 to split key value pair from a field called "data". Currently we need to migrate to logstash 6.8.0, Could someone help me for this?

Currently i'm using the below ruby code, it is not working properly

ruby {
code => "
event.get('data').each {|k, v|
event.set(k, v)
}
event.remove('data')
"
}

Example of data field is:
{"data":[{"key1":"value1","key2":"value2","key3":"value3", etc}]} Like this the log has key: value more than 20

2

That shows data is an array, so the .each will return the hash, not the key value pairs. You could try event.get('[data][0]').each

3 
ruby {
code => "
event.get('[data][0]').each {|k, v|
event.set(k, v)
}
event.remove('data')
"
}

Hi @Badger , i tried the above one and getting logs with _rubyexception
in nohup file i'm getting this

Ruby exception occurred: undefined method `each' for nil:NilClass

Also i came to know that we are receiving logs like this also

{"data":[{"key1":"value1","key2":"value2","key3":"value3", etc}, {}, {"key1":"value1","key2":"value2","key3":"value3", etc}]}

4

That is telling you that [data][0] does not exist. If it is not present on all events then you could try changing

event.get('[data][0]').each {|k, v|
    event.set(k, v)
}
event.remove('data')

to

d = event.get('[data][0]')
if d
    d.each {|k, v|
        event.set(k, v)
    }
    event.remove('data')
end
5 
ruby {
code => "
d = event.get('[data]')
if d
    d.each {|k, v|
        event.set(k, v)
    }
    event.remove('data')
end
"
}

Thanks @Badger This one is working fine, anyway to avoid empty '{}' log i have dropped it from LS itself.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.