# Parse JSON in "message" field

**URL:** <https://discuss.elastic.co/t/parse-json-in-message-field/267854>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [March 19, 2021, 7:52pm UTC](https://discuss.elastic.co/t/parse-json-in-message-field/267854 "2021-03-19T19:52:12Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![surprised\_ferret](https://avatars.discourse-cdn.com/v4/letter/s/dbc845/32.png) [@surprised\_ferret](https://discuss.elastic.co/u/surprised_ferret)\
**Post date:** [March 19, 2021, 7:52pm UTC](https://discuss.elastic.co/t/parse-json-in-message-field/267854/1 "2021-03-19T19:52:12Z")

</div>

Elastic version 7.11.1

- How do I parse a JSON structure (nested, one field has an array) out into separate fields of their own?

I'm running filebeat on my k8 instance, this is what my "filebeat.yml" value looks like, inside the filebeat-kubernetes.yaml file.

I think I am missing something but the documentation isn't very clear.

I have looked at [Filebeat parse json](https://discuss.elastic.co/t/filebeat-parse-json/130834) and [Filebeat JSON message](https://discuss.elastic.co/t/filebeat-json-message/106349) but they are using something called `filebeat.prospectors` which looks a bit like `filebeat.inputs` but different.

I have used the `decode_json_fields` key because I looked at this doc: [Decode JSON fields | Filebeat Reference [7.11] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/decode-json-fields.html)

```auto
  filebeat.yml: |-
    filebeat.inputs:
    - type: container
      paths:
        - /var/log/containers/*.log
      processors:
        - add_kubernetes_metadata:
            host: ${NODE_NAME}
            matchers:
            - logs_path:
                logs_path: "/var/log/containers/"
        - decode_json_fields:
            fields: ["message"]
            max_depth: 8

```

the message JSON i want to parse out looks like this

`	{"@timestamp":"2021-03-19T19:46:00.2675696+00:00","level":"Information","messageTemplate":"Executing ObjectResult, writing value of type '{Type}'.","message":"Executing ObjectResult, writing value of type '\"Microsoft.AspNetCore.Mvc.ProblemDetails\"'.","fields":{"Type":"Microsoft.AspNetCore.Mvc.ProblemDetails","EventId":{"Id":1,"Name":"ObjectResultExecuting"},"SourceContext":"Microsoft.AspNetCore.Mvc.Infrastructure.ObjectResultExecutor","ActionId":"caefff34-0ffd-4aa0-81a6-b68c86df21e5","ActionName":"etc_api.AcknowledgeTvlController.AcknowledgeTvl (etc-api)","RequestId":"0HM758EQLA55U:00000002","RequestPath":"/rest/v1/tvl/acknowledgement","SpanId":"4902e7163ebd8441","TraceId":"0c5a930c74bc7247b2c006002de2602b","ParentId":"0000000000000000","ConnectionId":"0HM758EQLA55U"}}`

---

<div class="post-metadata">

**Author:** ![Andre\_Letterer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andre_letterer/32/42248_2.png) [@Andre\_Letterer](https://discuss.elastic.co/u/Andre_Letterer)\
**Post date:** [March 20, 2021, 2:18pm UTC](https://discuss.elastic.co/t/parse-json-in-message-field/267854/2 "2021-03-20T14:18:42Z")

</div>

Hi,

Prospectors were famous in version 6.x and are now sunset a longer time ago towards inputs:

> **[Brewing in Beats: Rename Filebeat prospectors to inputs | Elastic Blog](https://www.elastic.co/blog/brewing-in-beats-rename-filebeat-prospectors-to-inputs)**

Additionally the `decode_json_fields` processor is not the right place as this would mean one field has containing json. But here the whole message is in JSON embedded and for that this particular documentation is the right place:

> **[Container input | Filebeat Reference \[7.11\] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-container.html#filebeat-input-container-config-json)**

So you could configure your snippet like that:

```auto
  filebeat.yml: |-
    filebeat.inputs:
    - type: container
      paths:
        - /var/log/containers/*.log
      json.add_error_key: true
      json.keys_under_root: true
      json.overwrite_keys: true
      processors:
        - add_kubernetes_metadata:
            host: ${NODE_NAME}
            matchers:
            - logs_path:
                logs_path: "/var/log/containers/"

```

And then add whatever json.xxxx settings you additionally might need.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 17, 2021, 4:19pm UTC](https://discuss.elastic.co/t/parse-json-in-message-field/267854/3 "2021-04-17T16:19:38Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.