discourse
(Discourse Support)
November 3, 2017, 8:29am
1
Based on the docs
You can configure Filebeat to dynamically reload prospector configuration files when there are changes. To do this, you specify a path (Glob) to watch for prospector configuration changes. When the files found by the Glob change, new prospectors are started/stopped according to changes in the configuration files.
What does the changes in the docs refer to? Does it mean new yml
files being added to the directory or changes within the existing files itself.
My current config looks something like that
filebeat.config_dir: /usr/share/filebeat/config
filebeat.config.prospectors:
path: /usr/share/filebeat/config/*.yml
reload.enabled: true
reload.period: 60s
and I store a single app_prospectors.yml
in /usr/share/filebeat/config
.
However, we run a service that generates a new app_prospectors.yml
based on the Docker containers that are currently running on the host every 60s. I noticed two things that seemed a little odd:
Filebeat doesn't pick up the new app_prospectors.yml
file if it was started when the file isn't present.
Also it does seem to me that changes in my app_prospectors.yml
file isn't picked up at all. Not sure how to debug this here
I'm running Filebeat v5.6.3.
Sample of my app_prospectors.yml
file
---
filebeat.prospectors:
- input_type: log
paths:
- "/var/docker/shared/dev/var-log/nginx/access.log*"
exclude_files:
- ".gz$"
document_type: nginx-access
- input_type: log
paths:
- "/var/docker/shared/dev/var-log/nginx/error.log*"
exclude_files:
- ".gz$"
document_type: nginx-error
- input_type: log
paths:
- "/var/docker/shared/dev/log/rails/production.log*"
exclude_files:
- ".gz$"
document_type: rails
exekias
(Carlos Pérez Aradros)
November 3, 2017, 10:16am
2
Hi @tgxworld ,
It should pick both new files and changes. Please review file permissions for your config
folder, they should be the same as in filebeat.yml
. Also check the log for details in case that doesn't help, paste it here if permissions don't help
discourse
(Discourse Support)
November 3, 2017, 12:48pm
3
Permission looks OK
-rwx------ 0 root root 469 Nov 2 09:29 filebeat.yml
drwx------ 2 root root 4096 Nov 3 12:43 config
-rw-r--r-- 1 root root 437 Nov 3 12:43 app_prospectors.yml
Here are the logs even after I deleted and added a new input_type
in app_prospectors.yml
2017/11/03 12:44:28.805873 reload.go:205: INFO Stopping 0 runners ...
2017/11/03 12:44:28.805884 reload.go:193: INFO Starting 0 runners ...
2017/11/03 12:44:58.468330 metrics.go:39: INFO Non-zero metrics in the last 30s: libbeat.config.reloads=1 libbeat.logstash.call_count.PublishEvents=6 libbeat.logstash.publish.read_bytes=36 libbeat.logstash.publish.write_bytes=5838 libbeat.logstash.published_and_acked_events=50 libbeat.publisher.published_events=50 publish.events=50 registrar.states.update=50 registrar.writes=6
2017/11/03 12:45:28.468405 metrics.go:39: INFO Non-zero metrics in the last 30s: libbeat.logstash.call_count.PublishEvents=6 libbeat.logstash.publish.read_bytes=36 libbeat.logstash.publish.write_bytes=4908 libbeat.logstash.published_and_acked_events=44 libbeat.publisher.published_events=44 publish.events=44 registrar.states.update=44 registrar.writes=6
2017/11/03 12:45:28.806190 reload.go:205: INFO Stopping 0 runners ...
2017/11/03 12:45:28.806229 reload.go:193: INFO Starting 0 runners ...
2017/11/03 12:45:58.468305 metrics.go:39: INFO Non-zero metrics in the last 30s: libbeat.config.reloads=1 libbeat.logstash.call_count.PublishEvents=6 libbeat.logstash.publish.read_bytes=36 libbeat.logstash.publish.write_bytes=7531 libbeat.logstash.published_and_acked_events=60 libbeat.publisher.published_events=60 publish.events=60 registrar.states.update=60 registrar.writes=6
If it picks up a new config, will it log the following output like how it did when I start Filebeat?
2017/11/03 02:47:28.493602 log.go:91: INFO Harvester started for file: /var/docker/shared/dev/var-log/nginx/error.log.1
exekias
(Carlos Pérez Aradros)
November 6, 2017, 12:37pm
4
Could you also check permissions inside config
folder?
discourse
(Discourse Support)
November 8, 2017, 6:23am
5
/usr/share/filebeat # ls -lah
total 15916
drwxr-xr-x 7 root root 4.0K Nov 6 07:09 .
drwxr-xr-x 10 root root 4.0K Nov 6 07:09 ..
-rw-r--r-- 1 root root 41 Oct 6 20:27 .build_hash.txt
-rw-r--r-- 1 root root 94.8K Oct 6 20:27 NOTICE
-rw-r--r-- 1 root root 811 Oct 6 20:27 README.md
drwxr-xr-x 2 root root 4.0K Nov 6 08:17 config
drwxr-x--- 2 root root 4.0K Nov 7 02:14 data
-rwxr-xr-x 1 root root 15.3M Oct 6 20:28 filebeat
-rw-r--r-- 1 root root 36.9K Oct 6 20:28 filebeat.full.yml
-rw-r--r-- 1 root root 24.5K Oct 6 20:28 filebeat.template-es2x.json
-rw-r--r-- 1 root root 19.6K Oct 6 20:28 filebeat.template-es6x.json
-rw-r--r-- 1 root root 19.6K Oct 6 20:28 filebeat.template.json
-rwx------ 1 root root 432 Nov 3 04:01 filebeat.yml
drwxr-xr-x 7 root root 4.0K Nov 3 02:40 module
drwxr-xr-x 2 root root 4.0K Nov 3 02:40 scripts
/usr/share/filebeat # ls -lah config/
total 16
drwxr-xr-x 2 root root 4.0K Nov 6 08:17 .
drwxr-xr-x 7 root root 4.0K Nov 6 07:09 ..
-rw-r--r-- 1 root root 576 Nov 6 08:17 app_prospectors.yml
-rw-r--r-- 1 root root 29 Nov 6 08:16 app_prospectors.yml.old
Here are the permissions.
exekias
(Carlos Pérez Aradros)
November 8, 2017, 11:30am
6
Permissions are wrong for:
Try running chmod go-r config/app_prospectors.yml
discourse
(Discourse Support)
December 5, 2017, 1:56am
7
Sorry for the late reply. Does this mean that the file should belong to the filebeat
user and only be readable/writable by the user only?
Because chmod go-r
would result in -rw-------
exekias
(Carlos Pérez Aradros)
December 11, 2017, 5:32pm
8
Yes, as a security measure we enforce that by default, you can change that behavior by using --strict.perms=false
argument, although that's discouraged.
Best regards
discourse
(Discourse Support)
December 12, 2017, 1:20am
9
Awesome thank you for the help!
discourse
(Discourse Support)
Closed
January 9, 2018, 1:20am
10
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.