Logstash does not parse agent from NGINX access logs as it already has filebeat agent information in agent

When tried to use default COMBINEDAPACHELOG for NGINX access logs, all fields are getting parsed correctly except agent field which contains useragent information. (https://www.elastic.co/guide/en/logstash/current/config-examples.html)

I have used filebeat to send events to logstash.
So logstash by default generates filebeat information in agent field.
It sees duplicate field name and ignores useragent information from NGINX access logs.

I used mutate filter to rename filebeat agent field to beatagent and then I was able to collect agent information correctly from access logs.
I am new to Logstash and wanted to know doing something like this is okay and shouldnt logstash have capability to differentiate between agent information from access logs and filebeat agent?

input {
beats {
port => 5044
filter {
mutate {
rename => {"agent" => "beatagent"}
grok {
match => [ "message" , "%{COMBINEDAPACHELOG}+%{GREEDYDATA:extra_fields}"]
overwrite => [ "message" ]
date {
match => [ "timestamp" , "dd/MMM/YYYY:HH:mm:ss Z" ]
remove_field => [ "timestamp" ]
output {
file {
path => "/var/log/output.txt"

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.