# Filebeat, ECS and fieds.\* namespace

**URL:** https://discuss.elastic.co/t/filebeat-ecs-and-fieds-namespace/255062
**Category:** Beats
**Tags:** ecs-elastic-common-schema, filebeat
**Created:** [November 11, 2020, 12:49pm UTC](https://discuss.elastic.co/t/filebeat-ecs-and-fieds-namespace/255062 "2020-11-11T12:49:16Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![A\_B](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/a_b/32/17104_2.png) [@A\_B](https://discuss.elastic.co/u/A_B)
#### Post date: [November 11, 2020, 12:49pm UTC](https://discuss.elastic.co/t/filebeat-ecs-and-fieds-namespace/255062/1 "2020-11-11T12:49:17Z")

</div>

Hello all,

I'm looking for some suggestions for the following situation.

Some context  
I'm trying to convince our developers to adopt ECS as their log format for container logs.

We run our containers in Kubernetes and use Filebeat as a DaemonSet to collect all container logs.

Filebeat config

```
- type: container
  fields:
    log_prefix: k8s
    log_idx: ${K8S_CLUSTER}
  fields_under_root: false
  multiline.pattern: '^[[:space:]]+(at|\.{3})\b|^Caused by:'
  multiline.negate: false
  multiline.match: after
  paths:
    - "/var/lib/docker/containers/*/*.log"
  processors:
    - add_kubernetes_metadata:
        in_cluster: true
    - decode_json_fields:
        add_error_key: true
        overwrite_keys: true
        fields: ["message"]
        target: ""
        when:
          equals:
            kubernetes.labels.logFormat: "ecs"

```

We are adding a couple of fields later used by Logstash to route the documents to the correct Elasticsearch indices.

For the `decode_json_fields` processor we have set `overwrite_keys` so that in the final document e.g. `message` is overwritten by the ECS `message` field from the container log.

This has worked quite well until the first logs containing `fields.*` came in and because of `overwrite_keys`, those fields over write the fields added by Filebeat.

I know I can put the additional fields Filebeat is adding in the root of the JSON. Is that the best option for me? I can't see `fields.*` as part of ECS. Does anyone know if it is?

Any other options for me that anyone can think of?

Cheers,  
AB

---

<div class="post-metadata">

### Author: ![ebeahan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ebeahan/32/78989_2.png) [@ebeahan](https://discuss.elastic.co/u/ebeahan)
#### Post date: [December 2, 2020, 5:23pm UTC](https://discuss.elastic.co/t/filebeat-ecs-and-fieds-namespace/255062/2 "2020-12-02T17:23:54Z")

</div>

You're correct that `fields.*` is not a defined fieldset in ECS.

Another possibility: use the `add_fields` processor and place your fields into a namespace that aligns with the ECS best practices for [custom fields](https://www.elastic.co/guide/en/ecs/current/ecs-custom-fields-in-ecs.html).

```auto
processors:
  - add_fields:
      target: My_Custom_Namespace
      fields:
        log_prefix: k8s
        log_idx: ${K8S_CLUSTER}

```

---

<div class="post-metadata">

### Author: ![A\_B](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/a_b/32/17104_2.png) [@A\_B](https://discuss.elastic.co/u/A_B)
#### Post date: [December 3, 2020, 12:05pm UTC](https://discuss.elastic.co/t/filebeat-ecs-and-fieds-namespace/255062/3 "2020-12-03T12:05:36Z")

</div>

Hi @ebeahan,

thank you very much for your reply and your suggestion.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 31, 2020, 2:05pm UTC](https://discuss.elastic.co/t/filebeat-ecs-and-fieds-namespace/255062/4 "2020-12-31T14:05:39Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
