So, I have been playing around with an ELK stack this week.
I've been able to track log files with filebeat, send them to logstash, send them on to elasticsearch and even view those in kibana.
Now, since I only have a single host, and the ELK stack and all services I want to fetch logs from are all running in docker.
The way I set this up is to have filebeat set on the host for easy access to all files. Now, I wanted to enable the nginx module in filebeat to not have to manually change things around and use the dashboards integrated in the module. Now, on the filebeat end, I think i got it to work.
Filebeat can see the log files, so that's good.
However. The only log files that reach kibana (or ES for that matter) are the ones I manually specified in the filebeat.yml.
I've been stuck on this stage for a day, and don't really know what to look into next. Any advice?
Also, the pipelines in logstash:
And the pipeline config I have set up in logstash:
