# Fields merging in 6.0 differs from 5.x

**URL:** https://discuss.elastic.co/t/fields-merging-in-6-0-differs-from-5-x/109838
**Category:** Beats
**Tags:** filebeat
**Created:** [November 30, 2017, 9:24pm UTC](https://discuss.elastic.co/t/fields-merging-in-6-0-differs-from-5-x/109838 "2017-11-30T21:24:11Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![tehmoon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tehmoon/32/24892_2.png) [@tehmoon](https://discuss.elastic.co/u/tehmoon)
#### Post date: [November 30, 2017, 9:24pm UTC](https://discuss.elastic.co/t/fields-merging-in-6-0-differs-from-5-x/109838/1 "2017-11-30T21:24:12Z")

</div>

Hi!

I was in the process of upgrade my filebeats from 5.6 to 6.0. But it broke my logstash pipeline. The reason is -- I think -- a bug in the merging fields process of the `fields` option in the yaml config file.

My configuration is as follow:

```auto
filebeat.prospectors:

- input_type: log
  paths:
    - /xxx/ansible_events/events.log
  fields:
    type: ansible_events

- input_type: log
  paths:
    - /xxx/xxx/log/main.log
  include_lines: ['^\[info\]'] # include info log
  exclude_lines: ['\)$', 'GET /users ', 'GET /ping '] # exclude lines that doesn't have the Sent information and also /users and /ping
  fields:
    type: cs_main

- input_type: log
  paths:
    - /var/log/syslog
    - /var/log/auth.log
    - /var/log/kern.log
  fields:
    type: syslog

name: filebeat

fields:
  env: production
  datacenter: yyyy
  group: zzzz

output.logstash:
  hosts: ["x.x.x.x:5044"]

path.data: /xxx/elastic_beats/filebeat/data
path.logs: /xxxx/elastic_beats/filebeat/log

```

I do all the sorting on the `fields.type` field. Everything worked fine in `5.6.4`, the files `syslog`,`auth.log` and `kern.log` would have `fields.type` set to `syslog`. `events.log` would have it set to `ansible_events` and `main.log` to `cs_main`.

After upgrading to `6.0.0` all my events were set to `syslog`. A quick debugging showed that `filebeat` was taking _the last_ `input_type` declared. To prove it, I just moved some section on top to the bottom and all of the events got that `fields.type`.

---

<div class="post-metadata">

### Author: ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [November 30, 2017, 11:02pm UTC](https://discuss.elastic.co/t/fields-merging-in-6-0-differs-from-5-x/109838/2 "2017-11-30T23:02:06Z")

</div>

Sounds like a bug and requires some investigation. Can you [report this bug on github](https://github.com/elastic/beats/issues) please?

---

<div class="post-metadata">

### Author: ![tehmoon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tehmoon/32/24892_2.png) [@tehmoon](https://discuss.elastic.co/u/tehmoon)
#### Post date: [December 1, 2017, 1:10am UTC](https://discuss.elastic.co/t/fields-merging-in-6-0-differs-from-5-x/109838/3 "2017-12-01T01:10:06Z")

</div>

Sure, I'll do it tomorrow.

Thank you for responding.

---

<div class="post-metadata">

### Author: ![tehmoon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tehmoon/32/24892_2.png) [@tehmoon](https://discuss.elastic.co/u/tehmoon)
#### Post date: [December 1, 2017, 10:19pm UTC](https://discuss.elastic.co/t/fields-merging-in-6-0-differs-from-5-x/109838/4 "2017-12-01T22:19:33Z")

</div>

I've opened the issue here: [https://github.com/elastic/beats/issues/5795](https://github.com/elastic/beats/issues/5795)

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 29, 2017, 10:19pm UTC](https://discuss.elastic.co/t/fields-merging-in-6-0-differs-from-5-x/109838/5 "2017-12-29T22:19:49Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
