# Clarifications regarding reload configuration

**URL:** <https://discuss.elastic.co/t/clarifications-regarding-reload-configuration/106284>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [November 3, 2017, 8:29am UTC](https://discuss.elastic.co/t/clarifications-regarding-reload-configuration/106284 "2017-11-03T08:29:19Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 3, 2017, 8:29am UTC](https://discuss.elastic.co/t/clarifications-regarding-reload-configuration/106284/1 "2017-11-03T08:29:19Z")

</div>

Based on [the docs](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-configuration-reloading.html)

> You can configure Filebeat to dynamically reload prospector configuration files when there are changes. To do this, you specify a path (Glob) to watch for prospector configuration changes. When the files found by the Glob change, new prospectors are started/stopped according to changes in the configuration files.

What does the changes in the docs refer to? Does it mean new `yml` files being added to the directory or changes within the existing files itself.

My current config looks something like that

```plaintext
filebeat.config_dir: /usr/share/filebeat/config

filebeat.config.prospectors:
  path: /usr/share/filebeat/config/*.yml
  reload.enabled: true
  reload.period: 60s

```

and I store a single `app_prospectors.yml` in `/usr/share/filebeat/config`.

However, we run a service that generates a new `app_prospectors.yml` based on the Docker containers that are currently running on the host every 60s. I noticed two things that seemed a little odd:

1. Filebeat doesn't pick up the new `app_prospectors.yml` file if it was started when the file isn't present.
2. Also it does seem to me that changes in my `app_prospectors.yml` file isn't picked up at all. Not sure how to debug this here 🤔

I'm running Filebeat v5.6.3.

#### Sample of my `app_prospectors.yml` file

```plaintext
---
filebeat.prospectors:
- input_type: log
  paths:
  - "/var/docker/shared/dev/var-log/nginx/access.log*"
  exclude_files:
  - ".gz$"
  document_type: nginx-access
- input_type: log
  paths:
  - "/var/docker/shared/dev/var-log/nginx/error.log*"
  exclude_files:
  - ".gz$"
  document_type: nginx-error
- input_type: log
  paths:
  - "/var/docker/shared/dev/log/rails/production.log*"
  exclude_files:
  - ".gz$"
  document_type: rails

```

---

<div class="post-metadata">

**Author:** ![exekias](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/exekias/32/28718_2.png) [@exekias](https://discuss.elastic.co/u/exekias)\
**Post date:** [November 3, 2017, 10:16am UTC](https://discuss.elastic.co/t/clarifications-regarding-reload-configuration/106284/2 "2017-11-03T10:16:27Z")

</div>

Hi @tgxworld,

It should pick both new files and changes. Please review file permissions for your `config` folder, they should be the same as in `filebeat.yml`. Also check the log for details in case that doesn't help, paste it here if permissions don't help 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 3, 2017, 12:48pm UTC](https://discuss.elastic.co/t/clarifications-regarding-reload-configuration/106284/3 "2017-11-03T12:48:14Z")

</div>

Permission looks OK

```auto
-rwx------ 0 root root 469 Nov 2 09:29 filebeat.yml
drwx------ 2 root root 4096 Nov 3 12:43 config
-rw-r--r-- 1 root root 437 Nov 3 12:43 app_prospectors.yml

```

Here are the logs even after I deleted and added a new `input_type` in `app_prospectors.yml`

```plaintext
2017/11/03 12:44:28.805873 reload.go:205: INFO Stopping 0 runners ...
2017/11/03 12:44:28.805884 reload.go:193: INFO Starting 0 runners ...
2017/11/03 12:44:58.468330 metrics.go:39: INFO Non-zero metrics in the last 30s: libbeat.config.reloads=1 libbeat.logstash.call_count.PublishEvents=6 libbeat.logstash.publish.read_bytes=36 libbeat.logstash.publish.write_bytes=5838 libbeat.logstash.published_and_acked_events=50 libbeat.publisher.published_events=50 publish.events=50 registrar.states.update=50 registrar.writes=6
2017/11/03 12:45:28.468405 metrics.go:39: INFO Non-zero metrics in the last 30s: libbeat.logstash.call_count.PublishEvents=6 libbeat.logstash.publish.read_bytes=36 libbeat.logstash.publish.write_bytes=4908 libbeat.logstash.published_and_acked_events=44 libbeat.publisher.published_events=44 publish.events=44 registrar.states.update=44 registrar.writes=6
2017/11/03 12:45:28.806190 reload.go:205: INFO Stopping 0 runners ...
2017/11/03 12:45:28.806229 reload.go:193: INFO Starting 0 runners ...
2017/11/03 12:45:58.468305 metrics.go:39: INFO Non-zero metrics in the last 30s: libbeat.config.reloads=1 libbeat.logstash.call_count.PublishEvents=6 libbeat.logstash.publish.read_bytes=36 libbeat.logstash.publish.write_bytes=7531 libbeat.logstash.published_and_acked_events=60 libbeat.publisher.published_events=60 publish.events=60 registrar.states.update=60 registrar.writes=6

```

If it picks up a new config, will it log the following output like how it did when I start Filebeat?

```plaintext
2017/11/03 02:47:28.493602 log.go:91: INFO Harvester started for file: /var/docker/shared/dev/var-log/nginx/error.log.1

```

---

<div class="post-metadata">

**Author:** ![exekias](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/exekias/32/28718_2.png) [@exekias](https://discuss.elastic.co/u/exekias)\
**Post date:** [November 6, 2017, 12:37pm UTC](https://discuss.elastic.co/t/clarifications-regarding-reload-configuration/106284/4 "2017-11-06T12:37:39Z")

</div>

Could you also check permissions inside `config` folder?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 8, 2017, 6:23am UTC](https://discuss.elastic.co/t/clarifications-regarding-reload-configuration/106284/5 "2017-11-08T06:23:57Z")

</div>

```plaintext
/usr/share/filebeat # ls -lah
total 15916
drwxr-xr-x 7 root root 4.0K Nov 6 07:09 .
drwxr-xr-x 10 root root 4.0K Nov 6 07:09 ..
-rw-r--r-- 1 root root 41 Oct 6 20:27 .build_hash.txt
-rw-r--r-- 1 root root 94.8K Oct 6 20:27 NOTICE
-rw-r--r-- 1 root root 811 Oct 6 20:27 README.md
drwxr-xr-x 2 root root 4.0K Nov 6 08:17 config
drwxr-x--- 2 root root 4.0K Nov 7 02:14 data
-rwxr-xr-x 1 root root 15.3M Oct 6 20:28 filebeat
-rw-r--r-- 1 root root 36.9K Oct 6 20:28 filebeat.full.yml
-rw-r--r-- 1 root root 24.5K Oct 6 20:28 filebeat.template-es2x.json
-rw-r--r-- 1 root root 19.6K Oct 6 20:28 filebeat.template-es6x.json
-rw-r--r-- 1 root root 19.6K Oct 6 20:28 filebeat.template.json
-rwx------ 1 root root 432 Nov 3 04:01 filebeat.yml
drwxr-xr-x 7 root root 4.0K Nov 3 02:40 module
drwxr-xr-x 2 root root 4.0K Nov 3 02:40 scripts
/usr/share/filebeat # ls -lah config/
total 16
drwxr-xr-x 2 root root 4.0K Nov 6 08:17 .
drwxr-xr-x 7 root root 4.0K Nov 6 07:09 ..
-rw-r--r-- 1 root root 576 Nov 6 08:17 app_prospectors.yml
-rw-r--r-- 1 root root 29 Nov 6 08:16 app_prospectors.yml.old

```

Here are the permissions.

---

<div class="post-metadata">

**Author:** ![exekias](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/exekias/32/28718_2.png) [@exekias](https://discuss.elastic.co/u/exekias)\
**Post date:** [November 8, 2017, 11:30am UTC](https://discuss.elastic.co/t/clarifications-regarding-reload-configuration/106284/6 "2017-11-08T11:30:04Z")

</div>

Permissions are wrong for:

> [@tgxworld](#):
>
> -rw-r--r-- 1 root root 576 Nov 6 08:17 app\_prospectors.yml

Try running `chmod go-r config/app_prospectors.yml`

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 5, 2017, 1:56am UTC](https://discuss.elastic.co/t/clarifications-regarding-reload-configuration/106284/7 "2017-12-05T01:56:32Z")

</div>

> [@exekias](#):
>
> Permissions are wrong for:

Sorry for the late reply. Does this mean that the file should belong to the `filebeat` user and only be readable/writable by the user only?

Because `chmod go-r` would result in `-rw-------`

---

<div class="post-metadata">

**Author:** ![exekias](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/exekias/32/28718_2.png) [@exekias](https://discuss.elastic.co/u/exekias)\
**Post date:** [December 11, 2017, 5:32pm UTC](https://discuss.elastic.co/t/clarifications-regarding-reload-configuration/106284/8 "2017-12-11T17:32:04Z")

</div>

> [@tgxworld](#):
>
> Sorry for the late reply. Does this mean that the file should belong to the filebeat user and only be readable/writable by the user only?

Yes, as a security measure we enforce that by default, you can change that behavior by using `--strict.perms=false` argument, although that's discouraged.

Best regards

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 12, 2017, 1:20am UTC](https://discuss.elastic.co/t/clarifications-regarding-reload-configuration/106284/9 "2017-12-12T01:20:48Z")

</div>

Awesome thank you for the help!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 9, 2018, 1:20am UTC](https://discuss.elastic.co/t/clarifications-regarding-reload-configuration/106284/10 "2018-01-09T01:20:54Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.