Filebeat default modules + custom log files

Elastic Stack Beats
1

Hi,

I'm playing around with filebeat, and was wondering if what I'm trying to do is correct/the way to go.

I'm running filebeat with some modules enabled (apache2, mysql & system).

On the one hand, I want to extract some more fields from apache2 (vhost, response_time, etc...), and on the other hand I want to include some custom application log file.

I know I can do the apache2 enrichment with a changed (or custom) ingest pipeline (see https://github.com/elastic/beats/pull/5861/files#diff-ef7f2010bd20958842a648c883e85958), and I will probably need to to something similar for the custom application log files.

The question now is how I configure filebeat to accommodate this.

Do I add some conditional pipelines including the filename?

output.elasticsearch:
  hosts: ["http://localhost:9200"]
  pipelines:
    - pipeline: "application_log"
      when.contains:
        source: "/path/to/file"
    - pipeline: "enhanced_apache"
      when.contains:
        fileset.module: "apache2"

Is this the way to go?

2

I'm looking in the module directory for apache2 (https://github.com/elastic/beats/tree/master/filebeat/module/apache2), and I can't find the pipeline config instruction.

Is the pipeline selection done via the pattern matching on the ingress node?

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.