Fields in newly created filebeat module

Im trying to create my own filebeat. I want to work with the direct elasticbeat ingest Pipeline (no Logstash in between).

I downloaded the filebeat development package from github and programmed my filebeat module. I made the GROK for the ingest pipeline tested it with the "_ingest/pipeline/_simulate" and with the make command i got the "mymodule/access/_meta/fields.yml".

When i copy this module now to an existing filebeat installation (copy the module folder to /usr/share/filebeat/modules/", newly created index indexes the data, but dont see the fields as i have defined them in my _meta/fields.yml.

The indexed document adds all fields as "keyword" and if i look at the "_template/filebeat-7.0.0" my definitions are also not there (i assume thats why my fields all get indexed as keyword).

Also if i delete all indicies from elasticseach and the "_template/filebeat-7.0.0" my fields dont show up after automatic recreation.

As it seems the Global "/etc/filebeat/fields.yml" is the only structure that is transferred to elasticseach upon first indexing from a filebeat.

Is there any way to add fields from my module to the index definition without modifying the Global "/etc/filebeat/fields.yml" or is there another way to acomplish this?

TIA Rolf

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.