Filebeat nginx include custom log field

Elastic Stack Beats
1

Nginx: Running latest nginx on Ubuntu. We have added some custom fields to the access log (e.g. server_name).

Filebeat: Just setup 7.4.1 and trying to use the filebeat nginx module to send the log files to our version 7.4.1 ELK. I have it all setup and it is working - kibana dashboard and all. However I do not know how to include the custom log fields?

How can I tweak the filebeat.yml to include the custom log field "server_name" above?

2

I think the new field should be in the message now, it's just not extracted. You could tcpdump some sample traffic to verify.

Look at the elasticsearch ingest pipelines that were loaded by the nginx module. I think you just add your new field to the grok patterns and add any additional processing you might need.

You can remove the

"remove": { "field": "message" } }

while you are developing the change so the entire message field will be kept, I've found that handy at times.

3

Great, got it, thanks for the reply!

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.