# Filebeat: filebeat.full.yml attribute duplication

**URL:** https://discuss.elastic.co/t/filebeat-filebeat-full-yml-attribute-duplication/51110
**Category:** Beats
**Created:** [May 27, 2016, 3:01am UTC](https://discuss.elastic.co/t/filebeat-filebeat-full-yml-attribute-duplication/51110 "2016-05-27T03:01:58Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![Jaren](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jaren/32/9986_2.png) [@Jaren](https://discuss.elastic.co/u/Jaren)
#### Post date: [May 27, 2016, 3:01am UTC](https://discuss.elastic.co/t/filebeat-filebeat-full-yml-attribute-duplication/51110/1 "2016-05-27T03:01:58Z")

</div>

It appears in the [filebeat.full.yml](https://github.com/elastic/beats/blob/master/filebeat/filebeat.full.yml) that their is some duplication in the below attributes.

fields:

- [https://github.com/elastic/beats/blob/master/filebeat/filebeat.full.yml#L61](https://github.com/elastic/beats/blob/master/filebeat/filebeat.full.yml#L61)
- [https://github.com/elastic/beats/blob/master/filebeat/filebeat.full.yml#L220](https://github.com/elastic/beats/blob/master/filebeat/filebeat.full.yml#L220)

fields\_under\_root:

- [https://github.com/elastic/beats/blob/master/filebeat/filebeat.full.yml#L69](https://github.com/elastic/beats/blob/master/filebeat/filebeat.full.yml#L69)
- [https://github.com/elastic/beats/blob/master/filebeat/filebeat.full.yml#L226](https://github.com/elastic/beats/blob/master/filebeat/filebeat.full.yml#L226)

If I set the second set `fields` and `fields_under_root` nothing happens ... to get the desired effect the first set needs to be set.

All and all a bit confusing ... took me a couple cycles to figure out i wasn't using the "second" set.

thanks in advance,

JG

---

<div class="post-metadata">

### Author: ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [May 27, 2016, 9:34am UTC](https://discuss.elastic.co/t/filebeat-filebeat-full-yml-attribute-duplication/51110/2 "2016-05-27T09:34:03Z")

</div>

These are not duplicates. There are different places in processing pipeline fields can be added. The first option is per prospector (one can configure multiple prospectors with potentially different fields). The second option is used by publisher pipeline ( I think ) and should add those fields to all events.

---

<div class="post-metadata">

### Author: ![Jaren](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jaren/32/9986_2.png) [@Jaren](https://discuss.elastic.co/u/Jaren)
#### Post date: [May 27, 2016, 5:13pm UTC](https://discuss.elastic.co/t/filebeat-filebeat-full-yml-attribute-duplication/51110/3 "2016-05-27T17:13:55Z")

</div>

Thanks - Based on your above comments if I added tags using the seconds set.... I should see those tags in source system (kafka in this case) . However, I see the reverse. I have to update the `tags` and `fields` in the first set to see addition in the output.

However, maybe I am looking for the tags in the wrong place? Maybe the seconds set doesn't impact the actually output data but something else?

---

<div class="post-metadata">

### Author: ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)
#### Post date: [May 27, 2016, 6:15pm UTC](https://discuss.elastic.co/t/filebeat-filebeat-full-yml-attribute-duplication/51110/4 "2016-05-27T18:15:31Z")

</div>

Could you provide an example of the configuration where tagging is not working as expected and an output event if possible.

The per prospectors `tags` should be appended to the global `tags` and added to each event.

The per prospector `fields` should be merged with the global `fields` and added to each event. The per prospector fields will take precedence over the global ones if there are conflicts.

---

<div class="post-metadata">

### Author: ![Jaren](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jaren/32/9986_2.png) [@Jaren](https://discuss.elastic.co/u/Jaren)
#### Post date: [May 29, 2016, 2:12am UTC](https://discuss.elastic.co/t/filebeat-filebeat-full-yml-attribute-duplication/51110/5 "2016-05-29T02:12:37Z")

</div>

Per your request ... config and output

```auto
filebeat.prospectors:
- input_type: log
  paths:
    - /var/log/nginx/access.log

  fields:
    role: flex
    env: production
  fields_under_root: true

name: stuff_by_jaren

tags: [
  "chance",
  "drake"
  ]

fields:
  nba: thunder

```

output:

```auto
{
  "@timestamp": "2016-05-29T01:54:12.753Z",
  "beat": {
    "hostname": "ip-X-X-X-X",
    "name": "ip-X-X-X-X"
  },
  "env": "production",
  "input_type": "log",
  "message": "127.0.0.1 - - [27/May/2016:18:44:07 -0400] \"GET / HTTP/1.1\" 404 19 \"-\" \"stuff here\"",
  "offset": 000000,
  "role": "flex",
  "source": "/var/log/nginx/access.log",
  "type": "log"
}

```

---

<div class="post-metadata">

### Author: ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)
#### Post date: [May 30, 2016, 8:52am UTC](https://discuss.elastic.co/t/filebeat-filebeat-full-yml-attribute-duplication/51110/6 "2016-05-30T08:52:28Z")

</div>

I just tried to reproduce this on master and couldn't. What is the exact version of filebeat you are using?

---

<div class="post-metadata">

### Author: ![Jaren](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jaren/32/9986_2.png) [@Jaren](https://discuss.elastic.co/u/Jaren)
#### Post date: [May 30, 2016, 6:06pm UTC](https://discuss.elastic.co/t/filebeat-filebeat-full-yml-attribute-duplication/51110/7 "2016-05-30T18:06:10Z")

</div>

I am using Alpha.

I can reproduce it on the same box. However, last night I installed on a different box and it worked as expected.

Odd... I will follow up if i see it again. thx

---

<div class="post-metadata">

### Author: ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)
#### Post date: [May 31, 2016, 8:54am UTC](https://discuss.elastic.co/t/filebeat-filebeat-full-yml-attribute-duplication/51110/8 "2016-05-31T08:54:05Z")

</div>

Ok, thanks. My first guess is normally a config file indentation issue, but your config above looks fine.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 5, 2017, 9:51pm UTC](https://discuss.elastic.co/t/filebeat-filebeat-full-yml-attribute-duplication/51110/9 "2017-07-05T21:51:21Z")

</div>


