Date filter treat the timestamp of the same log differently

Elastic Stack Logstash
1

Hello
I hava a problem about date filter.
My conf is below.

input {
	file { path => "/var/log/secure" }
}

filter {
	dissect { mapping => { "message" => "%{timestamp->} %{+timestamp} %{+timestamp} %{hostname} %{another}" } }

	date { match => [ "timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ] }
}

output {
	elasticsearch { hosts => "localhost:9200" }
}

This is a result of my conf. The november log was recorded in 2019. But The december log was recorded in 2018. Am i wrong? :frowning:

{
      "hostname" => "MHKG",
          "host" => "MHKG",
       "another" => "sshd[13361]: Received disconnect from ::ffff:192.168.244.18: 11: Disconnect requested by Windows SSH Client.\r",
       "message" => "Nov 30 23:44:36 Sensor sshd[13361]: Received disconnect from ::ffff:192.168.244.18: 11: Disconnect requested by Windows SSH Client.\r",
          "path" => "/var/log/secure",
    "@timestamp" => 2019-11-30T23:44:36.000Z,
     "timestamp" => "Nov 30 23:44:36",
      "@version" => "1"
}
{
      "hostname" => "MHKG",
          "host" => "MHKG",
       "another" => "sshd[2264]: Accepted password for root from 192.168.56.1 port 8246 ssh2\r",
       "message" => "Dec  1 02:38:21 Sensor sshd[2264]: Accepted password for root from 192.168.56.1 port 8246 ssh2\r",
          "path" => "/var/log/secure",
    "@timestamp" => 2018-12-01T02:38:21.000Z,
     "timestamp" => "Dec 1 02:38:21",
      "@version" => "1"
}
2

@e997cd7e8d9915436150, your input timestamp field does not explicitly set the year, so the date filter has to guess the year. In your case, it is correctly guessing that the event with the December timestamp field occurred last year in 2018. It is incorrectly guessing that the event with the November timestamp occurred this year in 2019. There are currently no settings that affect how the date filter guesses the year for timestamps that do not explicitly provide it. The best way to ensure that the year is correct for your events is to explicitly provide the year in the timestamp field.

3

Until last year, logs of 12 months were recorded in the same year.
But now, January to November recorded in 2019. And December recorded in 2018.
The same is true with past versions.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.