Filebeat Log Error

Elastic Stack Beats
1

Hello everyone!

I have an error with Filebeat... checking /var/log/filebeat/filebeat , I am seeing this error:

2018-08-27T17:37:09.165Z INFO instance/beat.go:273 Setup Beat: filebeat; Version: 6.4.0
2018-08-27T17:37:09.167Z INFO pipeline/module.go:98 Beat name: elk640.oracle.com
2018-08-27T17:37:09.168Z INFO [monitoring] log/log.go:114 Starting metrics logging every 30s
2018-08-27T17:37:09.168Z INFO instance/beat.go:367 filebeat start running.
2018-08-27T17:37:09.168Z INFO registrar/registrar.go:134 Loading registrar data from /var/lib/filebeat/registry
2018-08-27T17:37:09.169Z INFO registrar/registrar.go:141 States Loaded from registrar: 10
2018-08-27T17:37:09.169Z WARN beater/filebeat.go:371 Filebeat is unable to load the Ingest Node pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the Ingest Node pipelines or are using Logstash pipelines, you can ignore this warning.
2018-08-27T17:37:09.169Z INFO crawler/crawler.go:72 Loading Inputs: 1
2018-08-27T17:37:09.169Z INFO log/input.go:138 Configured paths: [/var/log/*.log]
2018-08-27T17:37:09.169Z INFO input/input.go:114 Starting input of type: log; ID: 11204088409762598069
2018-08-27T17:37:09.170Z INFO crawler/crawler.go:106 Loading and starting Inputs completed. Enabled inputs: 1
2018-08-27T17:37:09.170Z INFO cfgfile/reload.go:140 Config reloader started
2018-08-27T17:37:19.170Z **ERROR cfgfile/reload.go:213 Error loading config: invalid config: yaml: line 9: could not find expected ':'**
2018-08-27T17:37:19.173Z INFO log/input.go:138 Configured paths: [/var/log/auth.log* /var/log/secure*]
2018-08-27T17:37:19.174Z INFO log/input.go:138 Configured paths: [/var/log/messages* /var/log/syslog*]
2018-08-27T17:37:19.174Z INFO input/input.go:114 Starting input of type: log; ID: 300630154341581075
2018-08-27T17:37:19.174Z INFO input/input.go:114 Starting input of type: log; ID: 17766284131079967355
2018-08-27T17:37:19.176Z INFO log/harvester.go:251 Harvester started for file: /var/log/messages
2018-08-27T17:37:29.174Z **ERROR cfgfile/reload.go:213 Error loading config: invalid config: yaml: line 9: could not find expected ':'**
2018-08-27T17:37:39.171Z INFO [monitoring] log/log.go:141 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":10,"time":{"ms":17}},"total":{"ticks":40,"time":{"ms":47},"value":40},"user":{"ticks":30,"time":{"ms":30}}},"info":{"ephemeral_id":"21688425-e712-4fcb-99f4-98dcd570a709","uptime":{"ms":30022}},"memstats":{"gc_next":6460672,"memory_alloc":4861120,"memory_total":7688776,"rss":23183360}},"filebeat":{"events":{"added":14,"done":14},"harvester":{"open_files":1,"running":1,"started":1}},"libbeat":{"config":{"module":{"running":0},"reloads":2},"output":{"events":{"acked":5,"batches":2,"total":5},"read":{"bytes":12},"type":"logstash","write":{"bytes":1233}},"pipeline":{"clients":3,"events":{"active":0,"filtered":9,"published":5,"retry":4,"total":14},"queue":{"acked":5}}},"registrar":{"states":{"current":10,"update":14},"writes":{"success":11,"total":11}},"system":{"cpu":{"cores":2},"load":{"1":0.99,"15":0.51,"5":0.84,"norm":{"1":0.495,"15":0.255,"5":0.42}}}}}}
> 2018-08-27T17:37:39.175Z ERROR cfgfile/reload.go:213 Error loading config: invalid config: yaml: line 9: could not find expected ':'
> 2018-08-27T17:37:49.176Z ERROR cfgfile/reload.go:213 Error loading config: invalid config: yaml: line 9: could not find expected ':'
> 2018-08-27T17:37:59.177Z ERROR cfgfile/reload.go:213 Error loading config: invalid config: yaml: line 9: could not find expected ':'
2018-08-27T17:38:09.170Z INFO [monitoring] log/log.go:141 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":20,"time":{"ms":12}},"total":{"ticks":90,"time":{"ms":57},"value":90},"user":{"ticks":70,"time":{"ms":45}}},"info":{"ephemeral_id":"21688425-e712-4fcb-99f4-98dcd570a709","uptime":{"ms":60022}},"memstats":{"gc_next":6532256,"memory_alloc":3277496,"memory_total":21894736,"rss":3878912}},"filebeat":{"events":{"added":13,"done":13},"harvester":{"open_files":1,"running":1}},"libbeat":{"config":{"module":{"running":0},"reloads":3},"output":{"events":{"acked":13,"batches":9,"total":13},"read":{"bytes":60},"write":{"bytes":4191}},"pipeline":{"clients":3,"events":{"active":0,"published":13,"total":13},"queue":{"acked":13}}},"registrar":{"states":{"current":10,"update":13},"writes":{"success":9,"total":9}},"system":{"load":{"1":0.6,"15":0.5,"5":0.76,"norm":{"1":0.3,"15":0.25,"5":0.38}}}}}}
> 2018-08-27T17:38:09.177Z ERROR cfgfile/reload.go:213 Error loading config: invalid config: yaml: line 9: could not find expected ':'

I checked several time my filebeat.yml, logstash.yml, elasticsearch.yml and kibana.yml and I couldn't find any error in line 9... do you know how I could fix this error?

Thanks!

2

The error is not in filebeat.yml itself. It looks like filebeat is configured with config reloading. It complains about one of the files found in the reloading-directory to be invalid yaml.

3

that is correct, I enble the reloading config in my filebeat.yml

#============================= Filebeat modules ===============================

filebeat.config.modules:

Glob pattern for configuration loading

path: ${path.config}/modules.d/*.yml

Set to true to enable config reloading

reload.enabled: true

Period on which files under path should be checked for changes

#reload.period: 10s

So, if that is not an error... I assume this thread could be closed

4

It is an error. There is an invalid configuration file in your modules.d directory. Unfortunately the error message does not report the actual file that failed :frowning:

I created an issue on the actual error message being not very helpful: #8122.

5

Hi Steffen, ok got it... if it is useful i am going to paste the three files (.conf) that I am using in Logstash (and beats):

input.conf
cat /etc/logstash/conf.d/input.conf

input {
beats {
port => 5044
}
}

filter.conf
cat /etc/logstash/conf.d/filter.conf

filter {
if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:[%{POSINT:syslog_pid}])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
}
syslog_pri { }
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}

output.conf
cat /etc/logstash/conf.d/output.conf

output {
elasticsearch {
hosts => ["localhost:9200"]
sniffing => true
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}

I hope this could help to resolve the issue... THANKS!

6

... also I am adding the info that I have regarding my modules.d

output enable modules:

/usr/share/filebeat/bin/filebeat --path.config=/etc/filebeat/ modules list

Enabled:
    nginx
    system

Disabled:
apache2
auditd
elasticsearch
icinga
iis
kafka
kibana
logstash
mongodb
mysql
osquery
postgresql
redis
traefik

this is the content in NGINX:

cat nginx.yml

- module: nginx
  # Access logs
  access:
    enabled: true

    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    var.paths ["/path/to/log/nginx/access.log*"]

  # Error logs
  error:
    enabled: true

    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    var.paths: ["/path/to/log/nginx/error.log*"]

this is the content in SYSTEM:

cat system.yml

- module: system
  # Syslog
  syslog:
    enabled: true

    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    #var.paths:

    # Convert the timestamp to UTC. Requires Elasticsearch >= 6.1.
    #var.convert_timezone: false

  # Authorization logs
  auth:
    enabled: true

    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    #var.paths:

    # Convert the timestamp to UTC. Requires Elasticsearch >= 6.1.
    #var.convert_timezone: false
7

You can test yaml syntax in filebeat configuration files using some yaml linters. E.g. a web based linter can be found at http://www.yamllint.com.

The problem is in your nginx configuration. You are missing the : symbol at line 9.

Your config is:

    var.paths ["/path/to/log/nginx/access.log*"]

but it must be:

    var.paths: ["/path/to/log/nginx/access.log*"]
8

yes, you are right... there is a typo in that line!!

I appreciate your help and now all is working better... the error about Error loading config: invalid config: yaml: line 9: could not find expected ':' has been gone!

THANKS!!!

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.