another beginner mistake. Retry attempt of my yaml underneath.
I dug a little further into the configuration. I belief that the predefined modules do not look at the main /etc/filebeat/filebeat.yml file.
I found that for each (in this case Zeek) logfile within a module a separate connection.yml exists. This yaml file describes the way the fields are parsed and renamed by filebeat.
The second stage occurs at Elasticsearch by the Ingest Node Pipeline. This pipeline is created by filebeat during the setup and is created based on a template which is present in the /usr/share/zeek//ingest/pipeline.yml. In the second stage enrichment with GeoIP is done (amongst others).
My conclusion is that the enrichment with DNS reverse lookups should be added in the 2nd stage (but could in theory also be done during the first stage by Filebeat).
I noticed that the Ingest Pipeline at Elasticsearch is created in JSON which makes editing a little more difficult (at leat for me).
My Question remains what the recommended way is to make this adjustment, there seem to be multiple scenario's:
Scenario 1: Let the enrichment occur at Filebeat (stage1) by adding the additional processor to the connection.yml
Scenario 2: Let the enrichment occur at Elastic (stage 2) by adding the additional processor in Yaml to the ingest\pipeline.yml file on the filebeat machine and re-run the setup.
Scenario 3: Let the enrichment occur at Elastic (stage 2) by adding the additional processor in JSON to the Ïngest Node Pipeline".
If I do the adjustment as described in scenario 1 or 2, what happens when filebeat gets updated, will this overwrite my customizations?
# ================================= Processors =================================
- add_cloud_metadata: ~
- add_docker_metadata: ~
- add_kubernetes_metadata: ~
nameservers: ['192.168.1.1', '192.168.1.2']
# ================================== Logging ===================================