How Can I See the Inside of my Logstash Event Object in a Ruby Script?

[redapplesonly (RAO)]

Hi Logstash Jedi Masters,

I’m building my first Logstash service. I want my filter to be able to do a little processing based on the data flowing through LS.

My LS setup is pretty basic. Here’s my LS config file, simplified for this post:

input {
  kafka {
    ...stuff here...
  }
}

filter {
  ruby {
    # Send all events to an external Ruby script:
    path => "/home/me/ruby_scripts/myScript.rb"
  }
}

output {
  elasticsearch{
    ...stuff here...
  }
}

Pretty simple. The problem is, when “myScript.rb” is called, I need it to actually look inside the event object and do some basic processing on the data inside. Here’s the script:

#!/usr/bin/env ruby

def filter(event)
        puts "myScript.rb :: I got an event!  Data is:  "+event.getData()
        return [event]
end

Obviously this doesn’t work, because I don’t understand much about that Logstash event object. I’ve looked it up (here) and it looks like the object is just a wrapper for another embedded data object called a ConvertedMap:

public Event()
    {
        this.metadata = new ConvertedMap(10);
        this.data = new ConvertedMap(10);
        this.data.putInterned(VERSION, VERSION_ONE);
        this.cancelled = false;
        setTimestamp(Timestamp.now());
    }

I’m assuming the ConvertedMap is defined here…? And this, in turn, is a child class of IdentityHashMap, defined here?

And here you can see my problem. I know that when data is pulled into Logstash, that data is passed to my Ruby script as an event object. But I never defined the interior of the event object. Is there some way of determining what it is? And how I can access the actual data LS is passing along to ElasticSearch? Are there any tutorials anyone could recommend?

Thanks!

[Badger]

You can get a hash containing the contents of the event using event.to_hash

[redapplesonly (RAO)]

@Badger

.to_hash

Yes! Absolutely right! Thank you Badger, you saved my bacon.

I'm going to add a few notes to anyone who might be following in my footsteps. I found that given Badger's trick, I could convert my event into a hash, and then convert that hash into a string. Here's how it works...

Again, I have this as my filter in my LS config file:

filter {
  ruby {
    # Pass event to external Ruby script
    path => "/home/me/ruby_scripts/myScript.rb"
  }
}

Here's that external script:

#!/usr/bin/env ruby

def filter(event)
        puts "Your event as a string is :: "+(event.to_hash).to_s
        return [event]
end

That syntax took a little while to figure out, but it was worth it.

For other beginners like me, here are the resources I used. Two handy sites on Ruby for beginners are here and here. And for commands related to hashes in Ruby, look here.

Thank again Badger!