# Filebeat - There are conflicts between additional prospectors and reload function

**URL:** <https://discuss.elastic.co/t/filebeat-there-are-conflicts-between-additional-prospectors-and-reload-function/85891>\
**Category:** Beats\
**Created:** [May 16, 2017, 9:27am UTC](https://discuss.elastic.co/t/filebeat-there-are-conflicts-between-additional-prospectors-and-reload-function/85891 "2017-05-16T09:27:10Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Kirax00](https://avatars.discourse-cdn.com/v4/letter/k/43a26b/32.png) [@Kirax00](https://discuss.elastic.co/u/Kirax00)\
**Post date:** [May 16, 2017, 9:27am UTC](https://discuss.elastic.co/t/filebeat-there-are-conflicts-between-additional-prospectors-and-reload-function/85891/1 "2017-05-16T09:27:11Z")

</div>

When I use the additional prospectors function and reload function, the reload function is not working.

Filebeat read the additional prospector configurations in the config.FetchConfigs, which call the mergeConfigFiles.  
In the mergeConfigFiles, the code is :

```
	tmpConfig := struct {
		Filebeat Config
	}{}
	err := cfgfile.Read(&tmpConfig, file)
	if err != nil {
		return fmt.Errorf("Failed to read %s: %s", file, err)
	}

	config.Prospectors = append(config.Prospectors, tmpConfig.Filebeat.Prospectors...)

```

The structure decides that the additional.yml should be :  
``

```

filebeat.prospectors:
```

- input\_type: log  
paths:
  - d:\temp\test\log\test1\*.log

- input\_type: log  
paths:
  - d:\temp\test\log\test2\*.log

when I enable the reload function: filebeat.config.prospectors: path: config/\*.yml reload.enabled: true reload.period: 10s

I find that it doesn't working. I do the debug and check the libbeat/cfgfile/reload.go, and find a doubt.  
First, the watcher is OK, filebeat can catch the change of the file.  
Second, when the reloader read the yml file, return empty.  
check the code in cfgfile.Run:

```
		// Load all config objects
		configs := []*common.Config{}
		for _, file := range files {
			c, err := LoadList(file)
			if err != nil {
				logp.Err("Error loading config: %s", err)
				continue
			}

			configs = append(configs, c...)
		}

		debugf("Number of module configs found: %v", len(configs))

```

The structure []\*common.Config{} require the yml like this:  
``

```

```

- input\_type: log  
paths:
  - d:\temp\test\log\test1\*.log

- input\_type: log  
paths:
  - d:\temp\test\log\test2\*.log

Because of the filebeat.prospectors, it can't get anything.

Please help me to check the issue.  
If I do something wrong, please tell me. Thanks.

By the way, if only use the main filebeat.yml. the reload also doesn't work because of it can't get the configurations.

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [May 16, 2017, 11:24am UTC](https://discuss.elastic.co/t/filebeat-there-are-conflicts-between-additional-prospectors-and-reload-function/85891/2 "2017-05-16T11:24:28Z")

</div>

Do not use the former one requiring `filebeat.prospectors`. It's deprecated and will be removed in the future.

The new `path` settings only do support prospectors configuration in array style (without `filebeat.prospectors`). This is fully intentional and [documented this way](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-configuration-reloading.html).

---

<div class="post-metadata">

**Author:** ![Kirax00](https://avatars.discourse-cdn.com/v4/letter/k/43a26b/32.png) [@Kirax00](https://discuss.elastic.co/u/Kirax00)\
**Post date:** [May 16, 2017, 11:54am UTC](https://discuss.elastic.co/t/filebeat-there-are-conflicts-between-additional-prospectors-and-reload-function/85891/3 "2017-05-16T11:54:05Z")

</div>

Hi steffens, thanks for your reply. if I remove the filebeat.prospectors in the additional yaml, filebeat can't get the prospector configurations, as the reason I said above.  
I tried to modify the source code in config.mergeConfigFiles as below:

```
	tmpConfig := []*common.Config{}
	err := cfgfile.Read(&tmpConfig, file)
	if err != nil {
		return fmt.Errorf("Failed to read %s: %s", file, err)
	}

	config.Prospectors = append(config.Prospectors, tmpConfig...)

```

It work normally.

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [May 16, 2017, 12:13pm UTC](https://discuss.elastic.co/t/filebeat-there-are-conflicts-between-additional-prospectors-and-reload-function/85891/4 "2017-05-16T12:13:58Z")

</div>

Sorry, but I can't follow you. Why exactly did you modify these lines? Which file? Line number?

What's you actual on-disk file layout (where did you put which file)?

Any log/error message? Have you tried to run filebeat with `-d '*,config'`, to also print a json representation of the configuration?

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [May 16, 2017, 12:14pm UTC](https://discuss.elastic.co/t/filebeat-there-are-conflicts-between-additional-prospectors-and-reload-function/85891/5 "2017-05-16T12:14:29Z")

</div>

Which version of filebeat are you using? Could it be that you hit [https://github.com/elastic/beats/pull/4227](https://github.com/elastic/beats/pull/4227) ?

---

<div class="post-metadata">

**Author:** ![Kirax00](https://avatars.discourse-cdn.com/v4/letter/k/43a26b/32.png) [@Kirax00](https://discuss.elastic.co/u/Kirax00)\
**Post date:** [May 16, 2017, 12:39pm UTC](https://discuss.elastic.co/t/filebeat-there-are-conflicts-between-additional-prospectors-and-reload-function/85891/6 "2017-05-16T12:39:45Z")

</div>

@ruflin, I use the master branch two week ago in the github.  
@steffens , the source is :  
beats/filebeat/config/config.go  
line 91 ~ 99  
it unpack the file settings as struct {Filebeat Config}.  
I do the debug. if without filebeat.prospectors, it can get the yaml file, but return nothing after reading.  
additional.yml:  
``

```

```

- input\_type: log  
paths:
  - d:\temp\test\log\test2\*.log

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [May 17, 2017, 9:06am UTC](https://discuss.elastic.co/t/filebeat-there-are-conflicts-between-additional-prospectors-and-reload-function/85891/7 "2017-05-17T09:06:19Z")

</div>

The above PR was only merged 7 days ago. So you should rebase on the current master.

As you are working on your own fork: What other changes did you made? What are your plans in enhancing filebeat? Interested to hear more.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 6, 2017, 9:32am UTC](https://discuss.elastic.co/t/filebeat-there-are-conflicts-between-additional-prospectors-and-reload-function/85891/8 "2017-06-06T09:32:16Z")

</div>

This topic was automatically closed after 21 days. New replies are no longer allowed.
