Logstash Date does not match system

Elastic Stack Logstash
1

Hi everyone,

I have 2 logs below need to parse. The out put timestamp match time but does not match date. Can anyone help me review m logstash conf ?

Log message 1:
[13:35:32.727:]ipr_tcp_open_connect_i(): fail opening comm channel
Logstash conf :
input {
beats {
port => 5044
}
}
filter
{
grok {
match => { "message" => "[%{TIME:dts}%{DATA:unwanted}]%{GREEDYDATA:rest}" }
remove_field => [ "unwanted" ]
}
date {
match => [ "dts", "HH:mm:ss.SSSS"]
target => "@timestamp"
timezone => "UTC"
remove_field => [ "dts" ]
}
}
output {
stdout {
codec => rubydebug
}
}

Log 1 out put:

D11
D11759×551 12.7 KB

Log message 2:
[30@00:01:33.048:] ===> INFO: Next log is "/app/svacc_pr_ctm_em/ctm_em/log/cmsg_log.CMSGATE.20220330.0", time >00:01:33.048:<

Logstash conf 2:
input {
beats {
port => 5044
}
}
filter
{
grok {
match => { "message" => "[%{DATA:unwanted}%{TIME:dts}:]%{SPACE}===>%{SPACE}%{LOGLEVEL:lvl}:%{SPACE}%{GREEDYDATA:rest}" }
}
mutate{
remove_field => [ "unwanted" ]
}
date {
match => [ "dts", "HH:mm:ss.SSS" ]
target => "@timestamp"
timezone => "UTC"
remove_field => [ "dts" ]
}
}
output {
stdout {
codec => rubydebug
}
}
The out put 2:

D12
D121325×574 24.9 KB

Out put timestamp of 2 logs should match currrent date of my machine not "2024-01-01"

Thanks everyone

2

That string does not specify the year, month, or day. As a result the filter will provide defaults that you may not like.

3

Hi, i tried config below and timestamp still get "2024-01-..." year and month should match current but it is not. Can you look at my conf please

input {
beats {
port => 5044
}
}
filter {
grok {
match => {
"message" => "[%{MONTHDAY:day}@%{TIME:time}:] ===> %{LOGLEVEL:lvl}: %{GREEDYDATA:rest}"
}
}
mutate {
add_field => { "dts" => "%{day}@%{time}" }
remove_field => [ "day", "time" ]
}

date {
    match => [ "dts", "DD@HH:mm:ss.SSS" ]
    target => "@timestamp"
    timezone => "UTC"
}

}
output {
stdout {
codec => rubydebug
}
}

Out put :
{
"dts" => "30@00:01:33.048",
"input" => {
"type" => "log"
},
"tags" => [
[0] "beats_input_codec_plain_applied"
],

},
      "rest" => "      Next log is \"/app/svacc_pr_ctm_em/ctm_em/log/cmsg_log.CMSGATE.20220330.0\", time >00:01:33.048:<",
     "agent" => {
    "ephemeral_id" => "1b90f69c-59d4-416e-a69d-a536d6d473b0",
            "name" => "Nhan-PC",
            "type" => "filebeat",
         "version" => "8.13.2",
              "id" => "43e08710-79c0-49c9-9713-b29aa462c5d5"
},
     "event" => {
    "original" => "[30@00:01:33.048:] ===> INFO:       Next log is \"/app/svacc_pr_ctm_em/ctm_em/log/cmsg_log.CMSGATE.20220330.0\", time >00:01:33.048:<"
},
  "@version" => "1",
   "message" => "[30@00:01:33.048:] ===> INFO:       Next log is \"/app/svacc_pr_ctm_em/ctm_em/log/cmsg_log.CMSGATE.20220330.0\", time >00:01:33.048:<",
       "lvl" => "INFO",
       "log" => {
    "offset" => 0,
      "file" => {
        "path" => "C:\\Filebeat\\logs\\D12.txt"
    }
},
"@timestamp" => 2024-01-30T00:01:33.048Z,
       "ecs" => {
    "version" => "8.0.0"
}

}

4

Can you point to any documentation that says what the month and year will default to if they are not parsed from the date/time string? Why do you think they should match the current date?