Get year, month and date when not specified on log

Elastic Stack Logstash
1

Hi,

I'm facing with a particular issue, I'm trying to find the best and accurate way to set year/month/day to my input document

My input log only prints hour:minute:second, like example above:

16:00:00,840 blablabla...

And when my log closes from past hour, it changes its name
from => .log
to => .log_YYYY-MM-DD-HH

So in this case, i used a ruby code to get year, month and day, and used to concatenate with event "logtime" which is hour, minute and second from log hour

    ruby {
        code => "
            event.set('timestamp', [event.get('source').split('/')[-1].split('_')[-1][0..9], event.get('logTime').split(',')[0]].join(' '))
        "
    }

However, when log is current i don't have this information from source field (name of file), so i thought using current timestamp (YYYY-MM-DD) to concatenate from logTime field (hour from my logfile)

However, this may cause a wrong data information, for example, if input comes on hour 23:59:59, i may use current day (which is already next day) and input data from future and loses integrity from my base.

I checked on a lot of links, and my issue seems to be the same of this link (for me, i need something like nearest default) as described on this git issue

Does anybody knows how to workaround this?

Thanks!

2

Hi everyone,

I solved my issue using a ruby code (not so clean, but it's working ok)

    ruby {
        code => "
                event_hour = event.get('logTime').split(':')[0].to_i
                current_hour = Time.now.strftime('%H').to_i
                if current_hour < event_hour
                        event.set('timestamp', (Time.now - 86400).strftime('%Y-%m-%d') + ' ' + event.get('logTime'))
                else
                        event.set('timestamp', Time.now.strftime('%Y-%m-%d') + ' ' + event.get('logTime'))
                end
        "
    }
1 Like
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.