When you build a custom beat, the build process creates your config file by concatenating your custom
_meta/beat.yml and libbeat's
_meta/config.yml.tmpl. When it does this, any custom processors you've defined in
_meta/config.yml.tmpl are shadowed by the second
processors section containing
add_host_metadata: ~ and
- add_cloud_metadata: ~. This leaves the user with the inconvenient task of either removing that second
processors section after installation, or merging them with the important processors up top, which they really shouldn't have to do.
This happens in
beats/dev-tools/mage/config.go, and there doesn't appear to be any configuration that'll suppress the inclusion of that second
processors section, as the merge is just a simple file concat and the
.tmpl file conditionals don't have any option for leaving it out entirely.
My custom beat (
o365beat) includes processors in its config to do some important things, and we don't want to put it on the users to fix this issue. We're considering removing the second section later in the build, which will be tricky because we won't have any control of future changes to that file.
Would you be open to a pull request that conditionally excludes the entire template processors section from libbeat's
_meta/config.yml.tmpl? I couldn't find any issues/PRs referencing this behavior.
See this o365beat issue for more details.