VMWare Horizon Syslog

[ntwrk80 (Ben Story)]

I'm new to logstash and of course one of the first log sources I try to bring in is a weird one. The server is a VMWare Horizon ID server and it's sending some very weird timestamps.

2020.05.129 12:05:360 [hc] INFO: passwordAdapter.PasswordIdpAdapter [] Login: jdoe - SUCCESS

So far I know that the format is year.month.dayoftheyear(today is day 129) and then HH:MM but have no clue on the 360. Does anyone know of a way to grok and change this time stamp into normal parameters for ingestion?

Thanks for any help,

--
Ben Story
twitter.com/ntwrk80

Pack IT Forwarding - Paying it forward by sharing knowledge

[ntwrk80 (Ben Story)]

Ok so I've made it further. I'm trying now to get the field dayofyear converted to the day of the month. Seems I need Ruby for this, but I haven't been able to cobble together working code. Ideas?

input {
  udp {
    port => 5514
    type => syslog
  }
}

filter {

if [host] == "10.140.43.249" or [host] == "10.140.43.248" {
   grok {
        match => { "message" => "^%{YEAR:year}.%{MONTHNUM:month}.(?<dayofyear>\d\d\d)\s%{HOUR:hour}:%{MINUTE:minute}:(?<wtf>\d\d\d) %{SYSL\
OG5424SD:syslog} INFO: %{JAVACLASS:javaclass} %{GREEDYDATA:syslog_message}" }
      add_field => [ "received_at", "%{@timestamp}" ]
      add_field => [ "received_from", "%{host}" ]
      }
   ruby {
        init => 'require "date"'
        code => "event['day'] = Date.strptime(event['dayofyear'],'%d').to_s"

        }
}


}

output {
       stdout { codec => rubydebug }
       }

[ptamba]

day of year is represented with D as per the date filter plugin docs. i suppose the other three digits represents fractions of seconds? if so, you can update your grok capture the whole timestamp with a single grok expression , then parse it with date filter using yyyy-MM-D HH:mm:SSS

if you’re sure that’s day

[ntwrk80 (Ben Story)]

That made life much easier. Thank you for the help @ptamba. I ended up with the following:

filter {

if [host] == "10.140.43.249" or [host] == "10.140.43.248" {
   grok {
        match => { "message" => "^(?<datetime>\d+\.\d+\.\d+\s\d+:\d+:\d+) %{SYSLOG5424SD:syslog} INFO: %{JAVACLASS:javaclass} %{GREEDYDA\
TA:syslog_message}" }
      add_field => [ "received_at", "%{@timestamp}" ]
      add_field => [ "received_from", "%{host}" ]
      }
      date {
           timezone => ["America/Chicago"]
           match => ["datetime", "yyyy.MM.D HH:mm:SSS"]
          }
} else {

  if [type] == "syslog" {
    grok {
      match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:sys\
\
log_pid}\])?: %{GREEDYDATA:syslog_message}" }
      add_field => [ "received_at", "%{@timestamp}" ]
      add_field => [ "received_from", "%{host}" ]
    }
    date {
      match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
    }
  }
}
}

For others that may need the Ruby method, I also got that to work.

filter {

if [host] == "10.140.43.249" or [host] == "10.140.43.248" {
   grok {
        match => { "message" => "^%{YEAR:year}.%{MONTHNUM:month}.(?<dayofyear>\d\d\d)\s%{HOUR:hour}:%{MINUTE:minute}:(?<wtf>\d\d\d) %{SY\
SLOG5424SD:syslog} INFO: %{JAVACLASS:javaclass} %{GREEDYDATA:syslog_message}" }
      add_field => [ "received_at", "%{@timestamp}" ]
      add_field => [ "received_from", "%{host}" ]
      }

   ruby {
        init => 'require "date"'
        code => '
             datevar = Date.strptime(event.get("dayofyear"),"%j")
             event.set("day", datevar.strftime("%d"))
             '
        }
} else {

  if [type] == "syslog" {
    grok {
      match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:sys\
\
log_pid}\])?: %{GREEDYDATA:syslog_message}" }
      add_field => [ "received_at", "%{@timestamp}" ]
      add_field => [ "received_from", "%{host}" ]
    }
    date {
      match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
    }
  }
}
}