Hi everyone.
I´m trying to use the date filter without any luck. I want to create a new field that is the combination of the %{date} and %{time} in my grok and have this new field to have a date type value.
Any help is appreciated.
Config
input {
beats {
port => 5044
}
}
filter {
mutate {
gsub => ["message", "\x1B[([0-9]{1,2}(;[0-9]{1,2})?)?[m|K]", ""]
}
grok {
match => {
"message" => [ "%{DATE_US:date} %{TIME:time} %{IP}-%{WORD} (%{DATA}\ (%{DATA}) %{WORD} (%{DATA}) %{WORD} %{WORD} %{WORD}: (?[^*). %{WORD} %{WORD} %{WORD} %{WORD} %{NUMBER:device_id} %{WORD} %{WORD} %{IP:device_ip}" ]
}
}
mutate {
add_field => {"receive_at" => "%{date} %{time}"} }
date {
match => [ "receive_at", "MM/dd/yyyy HH:mm:ss" ]
target => "@timestamp"
remove_field => [ "receive_at", "date", "time" ]
}
if "_grokparsefailure" in [tags] {
drop { }
}
}
output {
elasticsearch {
hosts => "${ELASTICSEARCH_HOST_PORT}"
index => "%{[@metadata][beat]}-%{[@metadata][version]}"
user => "${ELASTIC_USERNAME}"
password => "${ELASTIC_PASSWORD}"
}
}
This is the date format in my logs that I´m trying to fix:
05/24/2021 17:21:04,073
I also tried with:
date {
match => [ "receive_at", "MM/dd/yyyy HH:mm:ss.SSS" ]
target => "@timestamp"
remove_field => [ "receive_at", "date", "time" ]
}
Output
"@timestamp": [
"2021-06-02T21:28:06.942Z"
"receive_at.keyword": [
"05/28/2021 21:26:20,957"
],