Help to calculate start time from disconnection time and session time

Elastic Stack Logstash
1

Hi all,

I'm trying to get the StartTime of a session from my icecast streaming logs, into the logs I has the Disconnection date and the session duration time.

In simple math need to be something like this:
new timestamp = event[@timestamp] - event['duration']*1000

But I can't made this works, my code that is not working is:

            date {
                  	match => [ "timestamp" , "dd/MMM/YYYY:HH:mm:ss Z" ]
            }
            ruby {
                  	init => "require 'time'"
                    code => "event['@timestamp'] = event['timestamp'] - (event['duration']*1000);"
            }

I got this error:

Ruby exception occurred: undefined method `-' for "08/Sep/2015:09:35:23 -0600":String {:level=>:error}

Thanks in advance for any advice!
Alejandro

2

The timestamp and @timestamp fields are strings. If you want to perform time math on them you have to parse them into a Time with e.g. Time.parse(). Be careful about timezone issues and converting back to the expected format for @timestamp. It might be easier to convert it to an epoch and use a separate date filter for getting that into @timestamp.

3

thanks @magnusbaeck, i'm really newbie with Ruby and I try to do this time convertion without success

This be the current code:
code => "event['timestamp_new'] = Time.parse(event['@timestamp']).to_i - event['duration']"

Some advice to understand what i'm doing wrong?

Thanks in advance!
Alejandro

4

In what way is it not working? Have you tried the code in a standalone Ruby interpreter like irb?

5

@magnusbaeck the error i got now is:

Ruby exception occurred: undefined method gsub!' for "2015-09-08T15:56:58.000Z":LogStash::Timestamp {:level=>:error}

I googled this error, but look really generic.

If I try this Date from IRB look that works well
irb(main):016:0> Time.parse('2015-09-08T15:56:58.000Z').to_i
=> 1441727818

Look like I has this error:

Some advice to fix? i can't understand what i need to change to fix...
Thanks

6

Ah, right. The @timestamp field contains a LogStash::Timestamp object rather than a string. Since that class overloads the subtraction operator I suspect you can just do

event['timestamp_new'] = event['@timestamp'] - event['duration']

provided that the duration field is an integer and not a string.

7

thanks!

If I try like the example:

event['timestamp_new'] = event['@timestamp'] - event['duration']

I got:
Ruby exception occurred: no implicit conversion to rational from nil {:level=>:error}

If I do:

event['timestamp_new'] = event['@timestamp'].to_i - event['duration'].to_i

Logstash start correctly and run, but don't insert any information into ES and don't show any error into logs and CPU use is really high...

8

I got:
Ruby exception occurred: no implicit conversion to rational from nil {:level=>:error}

Do all events have duration fields?

Logstash start correctly and run, but don't insert any information into ES and don't show any error into logs and CPU use is really high...

Increasing the logging verbosity with --verbose or --debug might help.