Date filter not working to set timestamp and receive logs

Elastic Stack Logstash
1

Hello community!

I'm facing a problem when trying to receive logs and using their date field as timestamp, following is my log example:
Dec 19 13:41:28 server-test smbd_audit: DOMAIN+user.fake|127.0.0.1|general|open|ok|r|Plan.xls

If I use the following filter, everything works fine and I can get the logs, but I still cannot set the timestamp as with the value of the new field added (since the original date doesn't have a year, a new field was created):

filter {
  grok {
    match => { "message" => [
      "%{SYSLOGTIMESTAMP:log_timestamp} %{SYSLOGHOST:server} %{WORD:type}: %{WORD:domain}\S%{USERNAME:user}\S%{IPV4:ip}\S%{WORD:folder}\S%{WORD:status}\S%{WORD:status2}\S%{WORD:action}\S%{GREEDYDATA:file}", 
      "%{SYSLOGTIMESTAMP:log_timestamp} %{SYSLOGHOST:server} %{WORD:type}: %{WORD:domain}\S%{USERNAME:user}\S%{IPV4:ip}\S%{WORD:folder}\S%{WORD:status}\S%{WORD:status2}"
      ]
    }
    add_field => ["new_timestamp", "%{log_timestamp} 2018"]
  }
}

As a solution to get the new field as my default timestamp, I used the date filter like below, however it doesn't work and even worse, I stop receiving logs.

filter {
  grok {
    match => { "message" => [
      "%{SYSLOGTIMESTAMP:log_timestamp} %{SYSLOGHOST:server} %{WORD:type}: %{WORD:domain}\S%{USERNAME:user}\S%{IPV4:ip}\S%{WORD:folder}\S%{WORD:status}\S%{WORD:status2}\S%{WORD:action}\S%{GREEDYDATA:file}", 
      "%{SYSLOGTIMESTAMP:log_timestamp} %{SYSLOGHOST:server} %{WORD:type}: %{WORD:domain}\S%{USERNAME:user}\S%{IPV4:ip}\S%{WORD:folder}\S%{WORD:status}\S%{WORD:status2}"
      ]
    }
    add_field => ["new_timestamp", "%{log_timestamp} 2018"]
  }
  date {
      match => [ "new_timestamp", "MMM  d HH:mm:ss YYYY", "MMM dd HH:mm:ss YYYY" ]
      timezone => "Etc/GMT" 
  }
}
2

Very interesting as this looks like it should work... have you tried looking at the output to stdout and seeing what the JSON looks like for new_timestamp and @timestamp?

3

Not yet, but I will try to do it. I just need to get some knowledge about how to do it first. Thank you!

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.