I have a logfile with some special logs and some general logs.I want the general logs to go in a general index and the special logs(logs which match any of my grok patterns) to go in a separate index and in the general index as well.
My pipeline conf looks like this:
input {
beats {
port => "5043"
}
}
filter {
grok {
match => [ "message", "PATTERN1",
"message", "PATTERN2"
]
}
if "_grokparsefailure" in [tags] {
mutate {
add_field => {"[@metadata][index]" => "generallogs"}
}
}
else{
mutate {
add_field => {"[@metadata][index]" => "speciallogs"}
}
}
}
output {
elasticsearch {
hosts => [ "localhost:9200" ]
index => "%{[@metadata][index]}"
}
}
The only problem with this is that special logs go only in the special log index and not in the general log index. Is there a solution ? Thanks !