# Oddness with decode\_json\_fields and json.keys\_under\_root

**URL:** <https://discuss.elastic.co/t/oddness-with-decode-json-fields-and-json-keys-under-root/132825>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [May 22, 2018, 1:00pm UTC](https://discuss.elastic.co/t/oddness-with-decode-json-fields-and-json-keys-under-root/132825 "2018-05-22T13:00:30Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![LollerAgent](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lolleragent/32/47486_2.png) [@LollerAgent](https://discuss.elastic.co/u/LollerAgent)\
**Post date:** [May 22, 2018, 1:00pm UTC](https://discuss.elastic.co/t/oddness-with-decode-json-fields-and-json-keys-under-root/132825/1 "2018-05-22T13:00:30Z")

</div>

Given the following configuration:

```auto
filebeat.prospectors:
- type: docker
  containers.ids:
    - ea97d09738e4bdf7b41cb926ab1276110ed3ce93f73f005d4f5da540ef0928c1
  processors:
  - add_docker_metadata: ~
  - decode_json_fields:
      fields: ["message"]
      target: "log"

```

The `message` field is decoded into the `log` (target) field, but the original `message` field remains as an encoded string at the root of the JSON object:

```auto
  "log": {
    "message": "",
    "level": "info",
    "distributed_trace_id": "",
    "http": {
      "request": {
        "remote_ip": "173.197.145.174",
        "method": "GET",
        "path": "/stylesheets/fonts/WhitneySSm-BookItalic.ttf",
        "host": ""
      },
      "response": {
        "status_code": 404
      }
    },
    "legacy_message": "ts=2018-05-21T20:48:51.880970898Z method=GET path=/stylesheets/fonts/WhitneySSm-BookItalic.ttf host= remote_ip=173.197.145.174"
  },
...
  "message": "{\"legacy_message\":\"ts=2018-05-21T20:48:51.880970898Z method=GET path=/stylesheets/fonts/WhitneySSm-BookItalic.ttf host= remote_ip=173.197.145.174\",\"message\":\"\",\"level\":\"info\",\"distributed_trace_id\":\"\",\"http\":{\"request\":{\"method\":\"GET\",\"path\":\"/stylesheets/fonts/WhitneySSm-BookItalic.ttf\",\"host\":\"\",\"remote_ip\":\"173.197.145.174\"},\"response\":{\"status_code\":404}}}",

```

If I add `json.keys_under_root: false` (which is `false` by default, right?) - the `message` field gets decoded underneath a new `json` key (not `log` as defined by the target) but the original `message` key gets deleted:

```auto
  "json": {
    "legacy_message": "ts=2018-05-21T20:46:41.807431938Z method=GET path=/stylesheets/fonts/WhitneySSm-BookItalic.ttf host= remote_ip=173.197.145.174",
    "message": "",
    "level": "info",
    "distributed_trace_id": "",
    "http": {
      "request": {
        "method": "GET",
        "path": "/stylesheets/fonts/WhitneySSm-BookItalic.ttf",
        "host": "",
        "remote_ip": "173.197.145.174"
      },
      "response": {
        "status_code": 404
      }
    }
  },

```

What is the correct behavior here? This seems odd. My goal is to get the decoded `message` field under a new `log` key, but delete the original `message` field that includes the encoded JSON. Is this possible? This is in 6.2.4.

---

<div class="post-metadata">

**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\
**Post date:** [May 22, 2018, 2:36pm UTC](https://discuss.elastic.co/t/oddness-with-decode-json-fields-and-json-keys-under-root/132825/2 "2018-05-22T14:36:57Z")

</div>

Indeed this behaviour is a bit odd, but this may be caused by some tricky behaviour of the different `json` parsing options, that are intended for different use cases.

For what you are trying to do, you can keep using the `decode_json_fields` option, and to delete the original `message` field add the [`drop_fields`](https://www.elastic.co/guide/en/beats/filebeat/6.2/drop-fields.html) processor, like this:

```auto
 - drop_fields:
     fields: ["message"]

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 19, 2018, 2:36pm UTC](https://discuss.elastic.co/t/oddness-with-decode-json-fields-and-json-keys-under-root/132825/3 "2018-06-19T14:36:59Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.