Specify extra fields in modules (e.g Elasticsearch)

[hunsw (Attila)]

I have several log type inputs in Filebeat, these specify one or more extra fields.

E.g. a.yml for logs from app 'A', b.yml for logs from app 'B'. The extra field 'index' is set to 'index_A' and index_B' respectively.

Logs are sent to a Logstash output where based on the field they end up in different Elasticsearch indices.

Now, I've just enabled the elasticsearch module on a test Elasticsearch node, but I cannot find a way to specify extra fields for logs harvested by this module.

Is there a way for this? I need to be able to separate logs on the Filebeat level, so when it arrives to Logstash it already has an index field.

[shaunak (Shaunak Kashyap)]

Every Filebeat module ends up starting a Filebeat input under the hood. And it's possible to override or set this input's settings when configuring the module. You can learn more about this capability here: https://www.elastic.co/guide/en/beats/filebeat/current/advanced-settings.html

One of the settings available to all inputs is fields, which allows you to set additional fields in events harvested by that input. You can learn more about this setting over here: https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-log.html#filebeat-input-log-fields.

Shaunak

[hunsw (Attila)]

Hi, thank you for your reply.

This indeed was the solution!