How do you remove (or not send) metadata from filebeat to logstash?

Elastic Stack Beats
1

Hi there. I am testing out filebeat on my Mac and it's successfully sending logs to logstash. Is there a way to only send the raw log message and not include any of the metadata?

For example, here's what I get once it's processed by logstash and written to an output file:

2019-09-25T18:21:31.177Z {name=myhostname.local, hostname=myhostname.local, id=x-x-x-x-x, os={name=Mac OS X, family=darwin, build=18G95, version=10.14.6, kernel=18.7.0, platform=darwin}, architecture=x86_64} 2019-09-25 12:21:29-06 myhostname softwareupdated[609]: Removing client SUUpdateServiceClient pid=32672, uid=0, installAuth=NO rights=(), transactions=0 (/usr/sbin/softwareupdate)

I don't need the info about my node -- I just want the log message as it originally existed. I don't know if it can be excluded on the filebeat side or if it needs to be filtered out at the logstash level.

Any suggestions would be greatly appreciated!

2

You can do it with processors (https://www.elastic.co/guide/en/beats/filebeat/current/drop-fields.html). Also you can comment out the add_host_metadata (see below) option.

processors:
  - drop_fields:
      fields: ["ecs.version", "agent.version", "agent.type", "agent.id", "agent.hostname", "input.type"]

  # - add_host_metadata: ~
  # - add_cloud_metadata: ~
3

That did the trick -- thank you very much!

4

Happy to help. Please close this topic by accepting the solution.

Thanks
Abhishek

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.