# Prospectors from config\_dir don't produce any harvesters

**URL:** <https://discuss.elastic.co/t/prospectors-from-config-dir-dont-produce-any-harvesters/37393>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [December 16, 2015, 6:52pm UTC](https://discuss.elastic.co/t/prospectors-from-config-dir-dont-produce-any-harvesters/37393 "2015-12-16T18:52:21Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Thomas\_Larsson\_Kron](https://avatars.discourse-cdn.com/v4/letter/t/9d8465/32.png) [@Thomas\_Larsson\_Kron](https://discuss.elastic.co/u/Thomas_Larsson_Kron)\
**Post date:** [December 16, 2015, 6:52pm UTC](https://discuss.elastic.co/t/prospectors-from-config-dir-dont-produce-any-harvesters/37393/1 "2015-12-16T18:52:21Z")

</div>

Hello,

I just started with filebeat and I encountered a problem.  
I want to feed my filebeat service with multiple config files and so I created a directory /etc/filebeat.d and copied the /etc/filebeat/filebeat.yml to /etc/filebeat.d/myapp.yml and edited it to point to a /tmplog/\*.log pattern.  
I configured the config\_dir in /etc/filebeat/filebeat.yml to point to /etc/filebeat.d.

I create a test logfile

```
echo "dummy message" >> /tmplog/apa.log

```

When I start the filebeat service I see this in the syslog:

```
...
Dec 16 18:38:58 localhost /usr/bin/filebeat[36889]: beat.go:107: Init Beat: filebeat; Version: 1.0.0
Dec 16 18:38:58 localhost /usr/bin/filebeat[36889]: config.go:129: Additional config files are fetched from: /etc/filebeat.d/
Dec 16 18:38:58 localhost /usr/bin/filebeat[36889]: config.go:107: Additional configs loaded from: /etc/filebeat.d/myapp.yml
Dec 16 18:38:58 localhost /usr/bin/filebeat[36889]: beat.go:133: filebeat sucessfully setup. Start running.
Dec 16 18:38:58 localhost /usr/bin/filebeat[36889]: registrar.go:66: Registry file set to: /var/lib/filebeat/registry
Dec 16 18:38:58 localhost /usr/bin/filebeat[36889]: registrar.go:76: Loading registrar data from /var/lib/filebeat/registry
Dec 16 18:38:58 localhost /usr/bin/filebeat[36889]: log.go:62: Harvester started for file: /var/log/auth.log
Dec 16 18:38:58 localhost /usr/bin/filebeat[36889]: spooler.go:77: Starting spooler: spool_size: 1024; idle_timeout: 5s
Dec 16 18:38:58 localhost /usr/bin/filebeat[36889]: log.go:62: Harvester started for file: /var/log/bootstrap.log
Dec 16 18:38:58 localhost /usr/bin/filebeat[36889]: log.go:62: Harvester started for file: /var/log/boot.log
Dec 16 18:38:58 localhost /usr/bin/filebeat[36889]: log.go:62: Harvester started for file: /var/log/cloud-init.log
Dec 16 18:38:58 localhost /usr/bin/filebeat[36889]: log.go:62: Harvester started for file: /var/log/cloud-init-output.log
Dec 16 18:38:58 localhost /usr/bin/filebeat[36889]: log.go:62: Harvester started for file: /var/log/iptables.log
Dec 16 18:38:58 localhost /usr/bin/filebeat[36889]: log.go:62: Harvester started for file: /var/log/daemon.log
Dec 16 18:38:58 localhost /usr/bin/filebeat[36889]: log.go:62: Harvester started for file: /var/log/mail.log
Dec 16 18:38:58 localhost /usr/bin/filebeat[36889]: log.go:62: Harvester started for file: /var/log/kern.log
Dec 16 18:38:58 localhost /usr/bin/filebeat[36889]: log.go:62: Harvester started for file: /var/log/yum.log
Dec 16 18:38:58 localhost /usr/bin/filebeat[36889]: log.go:62: Harvester started for file: /var/log/user.log
Dec 16 18:38:58 localhost /usr/bin/filebeat[36889]: crawler.go:78: All prospectors initialised with 11 states to persist
...

```

So no mention of any harvester picking up the log file from the pattern defined by myapp.yml.  
But there are no warnings or errors in the log that indicate if I have a bad configuration file.

If I just add the same pattern as a second path to the prospector in /etc/filebeat/filebeat.yml it works without any problems.

I have enabled debug level logging but found nothing of any use.

Can someone explain how I should continue to troubleshoot this?

Best Regards  
/Thomas

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [December 16, 2015, 7:21pm UTC](https://discuss.elastic.co/t/prospectors-from-config-dir-dont-produce-any-harvesters/37393/2 "2015-12-16T19:21:26Z")

</div>

Can you please post your configuration files.

---

<div class="post-metadata">

**Author:** ![Thomas\_Larsson\_Kron](https://avatars.discourse-cdn.com/v4/letter/t/9d8465/32.png) [@Thomas\_Larsson\_Kron](https://discuss.elastic.co/u/Thomas_Larsson_Kron)\
**Post date:** [December 16, 2015, 7:33pm UTC](https://discuss.elastic.co/t/prospectors-from-config-dir-dont-produce-any-harvesters/37393/3 "2015-12-16T19:33:26Z")

</div>

Oops,

Seems I had made some configuration error in the /etc/filebeat.d/myapp.yml file. I'm not sure what.  
I recopied the original filebeat.yml file and very carefully only changed the path setting, and now it worked.

So I won't bother posting my configuration. Thanks for the swift reply though!

I really wish for some feedback in the logs when there are errors in a configuration file and not just the kind of happy log output I got in this case.

Best Regards  
/Thomas

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 9:57pm UTC](https://discuss.elastic.co/t/prospectors-from-config-dir-dont-produce-any-harvesters/37393/4 "2017-07-05T21:57:27Z")

</div>


