# Filebeat field name collision with external prospectors?

**URL:** <https://discuss.elastic.co/t/filebeat-field-name-collision-with-external-prospectors/138809>\
**Category:** Beats\
**Created:** [July 6, 2018, 1:36am UTC](https://discuss.elastic.co/t/filebeat-field-name-collision-with-external-prospectors/138809 "2018-07-06T01:36:29Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![ceekay](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ceekay/32/5687_2.png) [@ceekay](https://discuss.elastic.co/u/ceekay)\
**Post date:** [July 6, 2018, 1:36am UTC](https://discuss.elastic.co/t/filebeat-field-name-collision-with-external-prospectors/138809/1 "2018-07-06T01:36:29Z")

</div>

After working through issues with the new host object in [Logstash errors after upgrading to filebeat-6.3.0](https://discuss.elastic.co/t/logstash-errors-after-upgrading-to-filebeat-6-3-0/135984/24) we've decided to go down the long road of namespacing our critical fields away from anything libbeat might put in the root in future.

However we've run in to an issue where nested fields with the same parent can't exist in both the main filebeat config and external prospector fragments.

E.g., using `test_namespace` as the parent object:

**/etc/filebeat/filebeat.yml**

```auto
filebeat:
  registry_file: /var/lib/filebeat/registry
  config.prospectors:
    enabled: true
    path: /etc/filebeat/conf.d/*.yml
fields:
  test_namespace.fqdn: civet.fqdn.xxxx
fields_under_root: true
output:
[...]

```

**/etc/filebeat/conf.d/authlog.yml**

```auto
- paths:
    - /var/log/auth.log
  type: log
  fields:
    test_namespace.container: syslog
  fields_under_root: true

```

**Output**

```auto
{
  "@timestamp": "2018-07-06T01:29:46.484Z",
  "@metadata": {
    "beat": "filebeat",
    "type": "doc",
    "version": "6.3.0"
  },
  "prospector": {
    "type": "log"
  },
  "input": {
    "type": "log"
  },
  "test_namespace": {
    "container": "syslog"
  },
  "beat": {
    "hostname": "civet",
    "version": "6.3.0",
    "name": "civet"
  },
  "host": {
    "name": "civet"
  },
  "source": "/var/log/auth.log",
  "offset": 67832,
  "message": "[redacted]"
}

```

Note that `test_namespace.fqdn` from filebeat.yml is missing from the output. If I rename the parent object in either of the configs, filebeat outputs both fields.

Is there any way to solve this, or is this a bug?

---

<div class="post-metadata">

**Author:** ![adrisr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adrisr/32/25423_2.png) [@adrisr](https://discuss.elastic.co/u/adrisr)\
**Post date:** [July 6, 2018, 10:17am UTC](https://discuss.elastic.co/t/filebeat-field-name-collision-with-external-prospectors/138809/2 "2018-07-06T10:17:42Z")

</div>

It does look like a bug to me, I will raise the issue.

I've found an ugly workaround, but probably not scalable if you have a lot of external prospectors.

in your prospector (same as before):

```auto
fields:
  test_namespace.container: syslog
  fields_under_root: true

```

in the main `fields` filebeat.yml use a different name:

```auto
fields:
  test_namespace_fqdn: civet.fqdn.xxxx # note the _ instead of .
  fields_under_root: true

```

then use a rename processor to move it into the required namespace:

```auto
processors:
- rename:
    fields:
     - from: 'test_namespace_fqdn'
       to: 'test_namespace.fqdn'
    ignore_missing: false
    fail_on_error: true

```

---

<div class="post-metadata">

**Author:** ![ceekay](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ceekay/32/5687_2.png) [@ceekay](https://discuss.elastic.co/u/ceekay)\
**Post date:** [July 6, 2018, 12:06pm UTC](https://discuss.elastic.co/t/filebeat-field-name-collision-with-external-prospectors/138809/3 "2018-07-06T12:06:26Z")

</div>

Thanks for the workaround, but if there's a fix coming, I'd rather wait. This is going to be packaged and automated so it all needs to be as clean as possible.

Do you have a link to the issue so I can subscribe to it?

---

<div class="post-metadata">

**Author:** ![adrisr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adrisr/32/25423_2.png) [@adrisr](https://discuss.elastic.co/u/adrisr)\
**Post date:** [July 6, 2018, 12:51pm UTC](https://discuss.elastic.co/t/filebeat-field-name-collision-with-external-prospectors/138809/4 "2018-07-06T12:51:27Z")

</div>

Here's the issue [https://github.com/elastic/beats/issues/7528](https://github.com/elastic/beats/issues/7528)

We are working on a fix, thanks for your report

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 3, 2018, 2:51pm UTC](https://discuss.elastic.co/t/filebeat-field-name-collision-with-external-prospectors/138809/5 "2018-08-03T14:51:28Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.