Calculate duration with ruby code

[Alisa_Faingold]

Hi all,
After several attempts to work with the "elapsed" filter to calculate the duration between events, I try to do so using Ruby code.
I can't quite figure out how I can get the time of a particular event based on its tag.
For example, these are my events:

05-07 13:20:48.808 1222 1236 D ShutdownThread: shutdown reason is: userrequested
05-07 10:22:28.306 657 657 I boot_progress_start: 9977
05-06 13:20:48.808 1222 1236 D ShutdownThread: shutdown reason is: userrequested
05-06 10:22:28.306 657 657 I boot_progress_start: 9977
05-05 13:20:48.808 1222 1236 D ShutdownThread: shutdown reason is: userrequested
05-05 10:22:28.306 657 657 I boot_progress_start: 9977

This is my code for now:
filter
{
dissect {
mapping => {
"message" => "%{[@metadata][timestamp]} %{+[@metadata][timestamp]} %{} %{} %{loglevel} %{msg}"
}
}
date{
match => [ "[@metadata][timestamp]", "MM-dd HH:mm:ss.SSS" ]
}

#device ON
if "boot_progress_start" in [message] {
	mutate {
		add_tag => "ON_event" 
	}
}

#device OFF
else if "shutdown reason is: userrequested" in [message] {
	mutate {
		add_tag => "OFF_event" 
	}
}

How my code in Ruby is supposed to show if I want to calculate the time between "ON" and "OFF" events.

Thanks!

[Badger]

You could try

ruby {
    code => '
        t = event.get("tags")
        if t.include? "ON_event"
            @start = event.get("@timestamp").to_f
        elsif t.include? "OFF_event"
            if @start
                event.set("elapsed", @start - event.get("@timestamp").to_f)
                @start = nil
            end
        end
    '
}

This assumes ordering of events is preserved, so you have to use '--pipeline.workers 1' and (for now) '--java-execution false'

[Alisa_Faingold]

I tried to use '--pipeline.workers 1' and '--java-execution false' dor the elapsed plugin but it doesn't help for me.

About the Ruby, If I have a few "ON" and "OFF" events and I want to calculate the duration for each such transaction, what needs to be changed?
Because it works great but it calculates the time between the first "ON" and the last "OFF".

Thank you!

[Badger]

@Alisa_Faingold

Because it works great but it calculates the time between the first "ON" and the last "OFF".

No, it calculates the elapsed time for each pair.

[Alisa_Faingold]

I added the same code (plus \3600 to get it in hours), and it is calculate for me the duration between the first "ON" and the last "OFF" (45.028 hours between 05-05 10:22 to 05-07 13:20)

[Badger]

That is what I would expect if you have pipeline.java_execution enabled. The three ON events are processed first, then the three OFF events, so only one of them gets an elapsed time added.

[Alisa_Faingold]

That make sense!
for some reason even when I add it to a configuration file I still don't get the wanted results, maybe you might share with me what your configuration file looks like?
Or, alternatively, how can I run that through the cmd?

Thank you very much!

[Badger]

Either add 'pipeline.java_execution: false' to logstash.yml or add '--java-execution false' to the command line.

[Alisa_Faingold]

Thank you!