Config template "processors" section shadows custom processors in custom beats

When you build a custom beat, the build process creates your config file by concatenating your custom _meta/beat.yml and libbeat's _meta/config.yml.tmpl. When it does this, any custom processors you've defined in _meta/config.yml.tmpl are shadowed by the second processors section containing add_host_metadata: ~ and - add_cloud_metadata: ~. This leaves the user with the inconvenient task of either removing that second processors section after installation, or merging them with the important processors up top, which they really shouldn't have to do.

This happens in beats/dev-tools/mage/config.go, and there doesn't appear to be any configuration that'll suppress the inclusion of that second processors section, as the merge is just a simple file concat and the .tmpl file conditionals don't have any option for leaving it out entirely.

My custom beat (o365beat) includes processors in its config to do some important things, and we don't want to put it on the users to fix this issue. We're considering removing the second section later in the build, which will be tricky because we won't have any control of future changes to that file.

Would you be open to a pull request that conditionally excludes the entire template processors section from libbeat's _meta/config.yml.tmpl? I couldn't find any issues/PRs referencing this behavior.

See this o365beat issue for more details.

This topic was automatically closed after 28 days. New replies are no longer allowed.