Error: Failed to start Filebeat sends log files to Logstash or directly to Elasticsearch

#1

Got following error when tried to start Filebeat service:

[root@VM1 filebeat]# systemctl start filebeat
Exiting: no modules or inputs enabled and configuration reloading disabled. What files do you want me to watch?
**Failed to start Filebeat sends log files to Logstash or directly to Elasticsearch**

[root@VM1 filebeat]# less /etc/filebeat/filebeat.yml
filebeat:
      paths:
        - /var/log/secure
        - /var/log/messages
        # - /var/log/*.log

      type: log
      enabled: true
      document_typ: syslog


output:
  logstash:
    hosts: ["172.27.95.196:5044"]
    bulk_max_size: 1024

    tls:
      certificate_authorities: ["/etc/pki/tls/certs/logstash-forwarder.crt"]

shipper:

logging:
  files:
    rotateeverybytes: 10485760 # = 10MB

[root@VM1 filebeat]# less /etc/filebeat/filebeat.reference.yml
#========================== Modules configuration =============================
filebeat.modules:

#-------------------------------- System Module --------------------------------
- module: system
  # Syslog
  syslog:
    enabled: true

I have installed Filebeat version 7.2.0 on a Red-Hat 7 box and can reach to the port 5044 of logstash.
On ELK server, I have installed Logstash (7.1.1), ElasticSearch(7.1.1) and Kibana and they are all listening on default ports.

I found the following error on /var/log/filebeat
ERROR instance/beat.go:877 Exiting: no modules or inputs enabled and configuration reloading disabled. What files do you want me to watch?

#2

I think this should be filebeat.inputs instead of just filebeat

https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-log.html

#3

I have changed it to filebeat.inputs but get the same error

filebeat.inputs:
      paths:
        - /var/log/secure
        - /var/log/messages
        # - /var/log/*.log

      type: log
      enabled: true
      document_typ: syslog

Error:
Failed to start Filebeat sends log files to Logstash or directly to Elasticsearch.
Exiting: no modules or inputs enabled and configuration reloading disabled. What files do you want me to watch?

I'm wondering why I can't see any new log in /var/log/filebeat

#4

Sorry to hear the issue is still there @Saber.Tala . Because filebeat is displaying errors that it does not know what files to watch, it leads me to believe that the error is in your filebeat.yml format.

Could you try modifying your filebeat.yml to match the default configuration?
It also looks like you have a typo "document_typ" instead of "document_type" and also, I believe document_type has been deprecated in Filebeat version 6.x and was used for the old filebeat.prospectors which have been replaced by inputs.
Please try to remove document_type, copy the same variables and order/formatting with dashes, so that "type" is at the top, "enabled" is below that, then "paths"

You can use this reference file to help, ctrl+f for "Filebeat inputs" to get to the input section:
https://www.elastic.co/guide/en/beats/filebeat/7.2/filebeat-reference-yml.html

Similar issue here: Filebeat file-god stopped and No modules or prospectors enabled and configuration reloading disabled

Hope this fixes the issue

2 Likes
#5

Thanks for your help, filebeat is now up and running, but I can't see any index in Kibana, from the machine that I installed fileabeat, I can telnet to Logstash over port 5044, however I can see flowing errors in /var/log/messages:

Aug  6 15:49:37 pfmwfdbs01 filebeat: 2019-08-06T15:49:37.041-0700#011INFO#011pipeline/output.go:105#011Connection to backoff(async(tcp://172.27.95.196:5044)) established
Aug  6 15:49:37 pfmwfdbs01 filebeat: 2019-08-06T15:49:37.074-0700#011ERROR#011logstash/async.go:256#011Failed to publish events caused by: lumberjack protocol error
Aug  6 15:49:37 pfmwfdbs01 filebeat: 2019-08-06T15:49:37.076-0700#011ERROR#011logstash/async.go:256#011Failed to publish events caused by: client is not connected

Any clue why filebeat doesn't send logs to Kibana/ElastichSearch?

#6

Great to hear it's up and running now @Saber.Tala! Hmm it might be a good idea to post your full configuration files for both filebeat and logstash but it seems like it might be an issue with the logstash output in your filebeat.yml

I don't have ssl/tls configured for my stack so I don't have any references, but it seems like you are again using very outdated configuration options.

According to the Logstash output plugin, the TLS option was last used in version 1.3:
https://www.elastic.co/guide/en/beats/filebeat/1.3/logstash-output.html

Please follow all the documentation for your specific version which is version 7.2 where it is now under SSL:
https://www.elastic.co/guide/en/beats/filebeat/7.2/logstash-output.html

More links:
Filebeat 7.2 SSL section
Filebeat 7.2 Output SSL settings
Logstash 7.1 Beats input

Look over the above links and make sure your filebeat ssl settings and logstash beats input settings are correct