We have a ton (200+) of applications that are all logging to the same index. Because of this, we are seeing a few field type collisions that cause messages to get bounced (to the tune of approximately 26 million bounced events per day).
I put together a quick ruby script in our main processing pipeline to handle these fields:
ruby {
code => '
fields = ["container", "destination", "host", "process", "source", "url", "user"]
fields.each { |f|
if event.include? f && event.get(f).is_a? String
event.set(f + "_val", event.get(f))
event.remove(f)
end
}
'
}
However, I'm getting the error:
(ruby filter code):6: syntax error, unexpected tCONSTANT\n...f && event.get(f).is_a? String\r\n
I'm sure this is something simple, but the black-box that is developing and debugging logstash pipelines has made it difficult for me, as I have a limited grasp of Ruby.
Can someone help out?