Hi guys,
Once again I have an issue trying out the ELK. In general, in Kibana I cannot see any tags I set and also my extra fields (e.g. number/date) displayed which I parsed. The message itself is seen in Kibana, line by line as a message but "tags" stay empty and also the parsed fields are not displayed.
And the index itself is just displayed as "[@metadata][index]".
Can you please advise? I am running version 6.2.4 if it is needed to know.
The log file looks like this:
22.01.2020 ! 08.39.55 ! 78 ! This is a text ! 0 #
22.01.2020 ! 08.39.55 ! 78 ! This as well ! 0 #
22.01.2020 ! 08.39.55 ! 78 ! And finally this ! 0 #
My part for filebeat.yml looks like this:
fields:
filebeat_type: abc-def
filebeat_base_path: "/path/to/our/log/"
filebeat_suffix_path: "with/suffix"
ignore_older: 24h
close_inactivhe: 5m
close_removed: true
close_eof: false
clean_removed: true
clean_inactive: 36h
And also, my logstash looks like this:
filter {
if [@metadata][logtype] == "abc-def"{
# see: https://www.elastic.co/blog/logstash-metadata or https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html
mutate {
add_field => {
"[@metadata][index]" => "my-test}"
}
}
grok {
match => {
"message" => "%{DATE_EU:date} ! %{DATA:time} ! %{INT:number} ! %{DATA:category} ! %{INT:occurence} #\n"}
tags => ["my_tag"]
mutate {
add_field => {"date" => "%{date}"}
add_field => {"time" => "%{time}"}
add_field => {"plant" => "%{number}"}
add_field => {"category" => "%{category}"}
add_field => {"occurence" => "%{occurence}"}
}
}
}