strange error with getting filebeat to work with the mysql module
Hi, I've been trying to get a filebeat running in order to parse a mysql slow log.
According to the tutorials online one simply has to configure the filebeat.yml file as such:
filebeat.inputs:
- module: mysql
slowlog:
enabled: true
var.paths: ["/var/db/mysql_data/<servername>-slow.log"]
setup.template.settings:
index.number_of_shards: 1
#index.codec: best_compression
#_source.enabled: false
name: <redacted>
tags: ["<redacted>"]
fields:
env: production
setup.dashboards.enabled: false
#============================== Template =====================================
setup.template.enabled: true
setup.template.name: "slowlogs-%{[agent.version]}"
setup.template.pattern: "slowlogs-%{[agent.version]}-*"
setup.template.fields: "${path.config}/fields.yml"
setup.template.overwrite: true
# Elasticsearch template settings
setup.template.settings:
index:
number_of_shards: 1
#codec: best_compression
#number_of_routing_shards: 30
# Enable ILM support. Valid values are true, false, and auto. When set to auto
# (the default), the Beat uses index lifecycle management when it connects to a
# cluster that supports ILM; otherwise, it creates daily indices.
setup.ilm.enabled: auto
setup.ilm.rollover_alias: "slowlogs"
setup.ilm.pattern: "{now/d}-000001"
setup.ilm.policy_name: "logrotate"
setup.ilm.overwrite: false
output.elasticsearch:
# Array of hosts to connect to.
hosts: ["<redacted>"]
# Optional protocol and basic auth credentials.
#protocol: "https"
#username: "elastic"
#password: "changeme"
processors:
- add_host_metadata: ~
#logging.level: debug
#monitoring.enabled: false
I've already removed the mysql error log since that logfile is empty and apparently filebeat doesn't like it when it's scraping an empty file.
module configs
I had initiall enabled the mysql module with the filebeat modules enable mysql command, but all that seems to do is rename a file in the modules.d directory, so I disabled it again.
The error
What happens when I start filebeat with filebeat -e is the following:
2019-12-02T12:10:01.375+0100 INFO [monitoring] log/log.go:154 Uptime: 45.006279ms
2019-12-02T12:10:01.375+0100 INFO [monitoring] log/log.go:131 Stopping metrics logging.
2019-12-02T12:10:01.375+0100 INFO instance/beat.go:432 filebeat stopped.
2019-12-02T12:10:01.375+0100 ERROR instance/beat.go:878 Exiting: Error while initializing input: No paths were defined for input accessing 'filebeat.inputs.0' (source:'/etc/filebeat/filebeat.yml')
Exiting: Error while initializing input: No paths were defined for input accessing 'filebeat.inputs.0' (source:'/etc/filebeat/filebeat.yml')
There's not really any debug options that make the error more clear.
According to the yml file (which is valid, I checked) the paths for the first module are correctly configured.
Additionally, the file it's trying to scrape exists in that location, so there's really no reason for it to act this way.
The question
How can I debug this issue?
I cannot for the life of me figure out what the problem is with this one vague error message and all tests returning with success.
Thank you!