@kvch
Please find below the config file. I think the reason why I get the error mentioned above is because I'm running filebeat and elasticsearch inside separate docker containers so filebeat doesn't have access to elasticsearch log path. Sharing a volume between the two containers might resolve the problem.
The problem is that Filebeat cannot find the elasticsearch/log fileset you configured. Is /usr/share/filebeat/module is accessible to Filebeat and contains the fileset?
Running the commands below inside the filebeat container indicates that the module filesets are accessible to filebeat:
[root@773170d8886b module]# pwd
/usr/share/filebeat/module
[root@773170d8886b module]# ls
apache2 elasticsearch iis kibana mongodb nginx postgresql system
auditd icinga kafka logstash mysql osquery redis traefik
[root@773170d8886b module]# ls elasticsearch/
audit deprecation gc module.yml server slowlog
[root@773170d8886b module]#
I shared a volume between elasticsearch (points to/usr/share/elasticsearch/logs) and filebeat (points to /var/log/elasticsearch) and I change the log fileset to the server fileset in the filebeat config
...
config:
- module: elasticsearch
server: # <----------------- Using server fileset
input:
type: docker
containers:
...
but the only logs being collected is for the gc fileset. I tried different filebsets such as audit, deprecation, slowlog but still I am only getting the gc fileset logs. Listing the content of where the elasticsearch are in filebeat container shows the following:
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.