How di I map Timestamp to event Timestamp from filebeat thru logstash

Elastic Stack Logstash
1

Hi,
I want to change the @timestamp to the timestamp out of event.original or message. I don't know what I do wrong.
right now the @timestamp is the time where the filebeat logfile ist importet thru logstash into elasticsearch.

@timestamp: Feb 15, 2023 @ 10:36:22.692
event.original: 06-02-2023 13:10:58:034, INFO , ....
message: 06-02-2023 13:10:58:034, INFO , ...

in my sample.conf I try it with:
filter {
date {
match => [ "Event Time", "dd-MM-YYYY HH:mm:ss:SSS" ]
target => "@timestamp"
}
}

Thank you

2

Are you using grok or dissect to parse the [Event Time] field from [message]? What does your configuration look like for that?

3

I think that's one of my problems or challenges. I entered Grok, but I suspect that this is where the error lies. I already apologize for my ignorance.

filter {
grok {
match => ["message", "%{SYSLOGBASE} %{GREEDYDATA:new_message}"]
}
date {
match => [ "Event Time", "dd-MM-YYYY HH:mm:ss:SSS" ]
target => "@timestamp"
}
}

4

This is not assigned, should be %{SYSLOGBASE:[@metadata][EventTime]}

date {
  match => [ "[@metadata][EventTime]", "dd-MM-YYYY HH:mm:ss:SSS" ]
  target => "@timestamp"
}
5

Also, the message format is not going to match SYSLOGBASE. That expects %{MONTH} +%{MONTHDAY} %{TIME}, whereas the [message] field appears to match ISO8601.

1 Like
6

Thank you for the info, but what or how do I have to enter it then?

7

%{DATESTAMP:[@metadata][EventTime]},%{SPACE}%{LOGLEVEL:loglevel}

8

I have now updated my Filter, but in Elasticsearch the cells are empty.

filter {
grok {
match => { "message" => %{DATESTAMP:[@metadata][EventTime]},%{SPACE}%{LOGLEVEL:loglevel, %{GREEDYDATA:message}"}
}
date {
match => [ "timestamp", "dd-MM-yyyy HH:mm:ss:SSS" ]
target => "@timestamp"
}
}

9

[@metadata][EventTime] in date:

filter {
grok {
  match => { "message" => "%{DATESTAMP:[@metadata][EventTime]},%{SPACE}%{LOGLEVEL:loglevel, %{GREEDYDATA:message}" }
}
date {
match => [ "[@metadata][EventTime]", "dd-MM-yyyy HH:mm:ss:SSS" ]
target => "@timestamp"
}

}
10

Thank you, that works :slight_smile:

1 Like
11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.