How to parse date field into @timestamp

Elastic Stack Logstash
1

i receive the spring app logs and i want to parse time from logs to @timestamp

that's what i got but timestamp is not the same.

input {
tcp {
port => 5000
codec =>plain
}
}

filter {
if [message] =~ /actions/ {
grok {
match => [ "message",
"%{GREEDYDATA:timestamp} %{LOGLEVEL:level} %{GREEDYDATA:data}"
]
}
date {
match => [ "timestamp" , "yyyy-MM-dd HH:mm:ss.SSS" ]
}
}
}

What am i doing wrong ?

2

Can you show us how does a full message look like?

3

Sorry i forgot the main string

target => "@timestamp"

P.S. To be clear tha part of config should be

date {
match => [ "timestamp" , "yyyy-MM-dd HH:mm:ss,SSS" ]
target => "@timestamp"
}

4

You need to share how your source message looks like, there is nothing wrong in the date filter, you also does not need to specify the target if the target is the @timestamp field, this is the default.

If your date filter is not work, so something is wrong with the value of your timestamp field, which could indicate something wrong with your parsing.

You need to share a sample of a message that you are receiving and trying to parse.

2 Likes
5

2024-01-16 20:45:31.293 DEBUG [actions,,] 8 --- [/api/v2/spans}}] o.s.w.HttpLogging : HTTP POST http://172.16.10.78:9411/api/v2/spans
2024-01-16 20:45:31.293 DEBUG [actions,,] 8 --- [/api/v2/spans}}] o.s.w.HttpLogging : Accept=[text/plain, application/json, application/*+json, /]
2024-01-16 20:45:31.294 DEBUG [actions,,] 8 --- [/api/v2/spans}}] o.s.w.HttpLogging : Writing [[B@5d1ee735] as "application/json"
2024-01-16 20:45:31.295 DEBUG [actions,,] 8 --- [/api/v2/spans}}] o.s.w.HttpLogging : Response 202 ACCEPTED
2024-01-16 20:45:40.001 INFO [actions,,] 8 --- [ scheduling-1] r.i.d.a.s.DeviceQueueExecutor : =========== runner start ===========
2024-01-16 20:45:40.046 DEBUG [actions,,] 8 --- [ scheduling-1] o.s.w.r.f.c.ExchangeFunctions : [25d86162] HTTP GET http://172.16.10.77:8001/?device=module&action=getStatus
2024-01-16 20:45:40.048 DEBUG [actions,,] 8 --- [or-http-epoll-4] o.s.w.r.f.c.ExchangeFunctions : [25d86162] [4bc96d38-1, L:/192.168.190.58:36332 - R:172.16.10.77/172.16.10.77:8001] Response 200 OK

6

The timestamp in your log has the format yyyy-MM-dd HH:mm:ss.SSS, but in your date filter you have yyyy-MM-dd HH:mm:ss,SSS.

The separator between seconds and miliseconds is not correct in your date filter.

1 Like
7

You are right! Thank! My fault!

The right part is

date {
match => [ "timestamp" , "yyyy-MM-dd HH:mm:ss.SSS" ]

target => "@timestamp"

}

8

I wanted to show that string
target => "@timestamp"
is commented, but # makes it bigger))

9

From the kubernetes pod console i se there is

.

But at the

journalctl -o cat -xefu logstash.service

i see there is

,

why is it so ?

10

This will show the logs from your logstash service, it is unrelated to the logs of your application that you are parsing.

The timestamp in logstash logs uses , between the seconds and miliseconds.

1 Like
11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.