Filebeat file-god stopped and No modules or prospectors enabled and configuration reloading disabled

caoimhe · April 23, 2018, 4:28pm

Service featbeat start exits with message "Exiting: No modules or prospectors enabled and configuration reloading disabled. What files do you want me to watch?"
Service filebeat status returns filebeat-god has stopped.
Service filebeat stop fails

How do I enable modules or prospectors and enable configuration reloading?
Is this linked to the file-god issue?

Included below is the error messages I'm receiving and my filebeat.yml

Thanks,
Caoimhe

service filebeat start

Starting filebeat: 2018/04/19 15:41:14.111768 beat.go:297: INFO Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/data/filebeat] Logs path: [/var/log/filebeat]
2018/04/19 15:41:14.111814 beat.go:192: INFO Setup Beat: filebeat; Version: 5.6.2
2018/04/19 15:41:14.111855 publish.go:228: WARN Support for loading more than one output is deprecated and will not be supported in version 6.0.
2018/04/19 15:41:14.111903 metrics.go:23: INFO Metrics logging every 30s
2018/04/19 15:41:14.112080 output.go:258: INFO Loading template enabled. Reading template file: /etc/filebeat/filebeat.template.json
2018/04/19 15:41:14.113092 output.go:269: INFO Loading template enabled for Elasticsearch 2.x. Reading template file: /etc/filebeat/filebeat.template-es2x.json
2018/04/19 15:41:14.114016 output.go:281: INFO Loading template enabled for Elasticsearch 6.x. Reading template file: /etc/filebeat/filebeat.template-es6x.json
2018/04/19 15:41:14.114926 client.go:128: INFO Elasticsearch url: http://localhost:9200
2018/04/19 15:41:14.114975 outputs.go:108: INFO Activated elasticsearch as output plugin.
2018/04/19 15:41:14.115027 logstash.go:90: INFO Max Retries set to: 3
2018/04/19 15:41:14.115112 outputs.go:108: INFO Activated logstash as output plugin.
2018/04/19 15:41:14.115228 publish.go:300: INFO Publisher name: sd-05cf-0367
2018/04/19 15:41:14.115406 async.go:63: INFO Flush Interval set to: 1s
2018/04/19 15:41:14.115421 async.go:64: INFO Max Bulk Size set to: 50
2018/04/19 15:41:14.115480 async.go:63: INFO Flush Interval set to: 1s
2018/04/19 15:41:14.115489 async.go:64: INFO Max Bulk Size set to: 2048
2018/04/19 15:41:14.115717 beat.go:346: CRIT Exiting: No modules or prospectors enabled and configuration reloading disabled. What files do you want me to watch?
Exiting: No modules or prospectors enabled and configuration reloading disabled. What files do you want me to watch?

service filebeat status
filebeat-god is stopped

service filebeat stop
Stopping filebeat: [FAILED]

cat filebeat.yml

filebeat.prospectors:
 paths:
  - /etc/TFSlogs/*.log
 input_type: log
 enabled: true
 document_type: log

  #-C:\ProgramData\Microsoft\Team Foundation\Server Configuration\Logs\*
  #- /var/log/messages
  #- c:\programdata\elasticsearch\logs\*
 
fields:
 environment: dev
 hosting_env: citi
 log_type: syslog
#================================ Outputs =====================================

Configure what outputs to use when sending the data collected by the beat
#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
 hosts: ["localhost:9200"]


#----------------------------- Logstash output --------------------------------
output.logstash:
 hosts: ["localhost:5044"]

loadbalance: true

#================================ Logging =====================================

logging.to_files: true
logging.to_syslog: false

harshbajaj16 · April 24, 2018, 6:48am

Hi caoimhe,

It seems your config file have some error as you defined enabled: true after paths: which is not correct.

Solution :
could you please try with below format :

filebeat.prospectors:

type: log
enabled: true
paths:

/etc/TFSlogs/*.log

Thanks,
Harsh Bajaj

caoimhe · April 24, 2018, 8:34am

Hi Harsh,

Thanks for the suggestion. This fixed it and filebeat is now running, however my Kibana is not showing the logs from the path: /etc/TFSlogs/*.log. Instead it is displaying old log files from the path /var/log/messages.

Do you have any idea why this might be?

Thanks,
Caoimhe

harshbajaj16 · April 24, 2018, 9:01am

Hi,

There might be so many reasons :

You can check with below suggestions :

Please check either you have applied any filter in kibana or check whether latest logs are coming from filebeat to logstash and also check elasticsearch index.

Restart services and check but first you should go with above suggestion. If your services are working fine then your latest logs should be visible in kibana (check kibana dashboard settings).

Thanks,
Harsh Bajaj

caoimhe · April 24, 2018, 1:24pm

Hi,

Thanks for your help! I got it working live using -var/log/messages. However, I have static log files in - /etc/TFSlogs are not being read.

I'm not sure that I'm using the index in the correctly in logstash.

[Dev root @ sd-05cf-0378 /etc/logstash/conf.d]
cat logstash-simple.conf

input{
beats{
port=>5044
}
}

output {
elasticsearch{
index=>"logstash-simple.conf"
template_overwrite=>"true"
template_name=>"logstash-simple.conf"
hosts=>["http://localhost:9200"]
}
}

Cheers,
Caoimhe

harshbajaj16 · April 25, 2018, 4:03am

Hi Caoimhe,

Could you please share the logs of filebeat and also share the config file of filebeat so that we can identify the issue.

Log file path : /var/log/filebeat/
Config file : /etc/filebeat/filebeat.yml

Thanks,
Harsh Bajaj

caoimhe · April 25, 2018, 9:08am

Hi,

I got it connected, I forgot to add the /* to the path -/etc/TFSlogs/*.
Thanks for all your help

This is my filebeat.yml file which worked:

Filebeat.yml

# cat filebeat.yml
#=========================== Filebeat prospectors =============================

filebeat.prospectors:
- type: log
  enabled: true
  paths:
  - /etc/TFSlogs/*
 #- /var/log/messages
 
fields:
  environment: dev
  hosting_env: citi
  log_type: syslog
#================================ Outputs =====================================

# Configure what outputs to use when sending the data collected by the beat
#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
  hosts: ["localhost:9200"]


#----------------------------- Logstash output --------------------------------
output.logstash:
  hosts: ["localhost:5044"]

#loadbalance: true

#================================ Logging =====================================

logging.to_files: true
#logging.to_syslog: false

Would the formatting of the log file account for why it would not be pick up?

system · May 23, 2018, 9:08am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Filebeat config error Beats filebeat	4	5398	August 9, 2018
Processors not working Beats filebeat	3	1114	March 27, 2018
Filebeat config prospectors not working in 6.0 Beats filebeat	7	4976	December 21, 2017
Filebeat prospector enable / disabled being ignored Beats filebeat	2	579	October 4, 2018
Filebeat 6.0 not loading the data file Beats filebeat	3	4562	December 22, 2017

Filebeat file-god stopped and No modules or prospectors enabled and configuration reloading disabled

Related topics