# How to parse json from message field

**URL:** <https://discuss.elastic.co/t/how-to-parse-json-from-message-field/350769>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [January 10, 2024, 3:15pm UTC](https://discuss.elastic.co/t/how-to-parse-json-from-message-field/350769 "2024-01-10T15:15:50Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![wrender1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wrender1/32/130685_2.png) [@wrender1](https://discuss.elastic.co/u/wrender1)\
**Post date:** [January 10, 2024, 3:15pm UTC](https://discuss.elastic.co/t/how-to-parse-json-from-message-field/350769/1 "2024-01-10T15:15:50Z")

</div>

Hello, I'm looking for help with parsing json data out of a log field. I'm using the elastic agent standalone in Kubernetes and not sure how to configure it for this. I believe the filebeat portion of the elastic agent is indexing the messages. Some of the fields I would like to index as a numeric integer if possible. Right now the events come in as something like this from the stdout of the containers and are pretty much useless in Kibana to build visualizations around:

```auto
  "message": [
    "I0104 15:47:24.388872 118 SyslogReporter.cpp:10] APP-METRIC {\"fields\":{\"drop\":0},\"measurement\":\"core.datafield\",\"tags\":{\"host\":\"blahblah-testing-9rtvp\",\"instance\":\"0\",\"source\":\"core.datafield\",\"type\":\"data\",\"worker\":\"0\"},\"time\":\"2024-01-04T15:47:24.000000000Z\"}"

```

Ideally I want to match based on "APP-METRIC" and strip out the json below the "fields" key.

Thanks!

---

<div class="post-metadata">

**Author:** ![jessgarson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jessgarson/32/129841_2.png) [@jessgarson](https://discuss.elastic.co/u/jessgarson)\
**Post date:** [January 11, 2024, 8:13pm UTC](https://discuss.elastic.co/t/how-to-parse-json-from-message-field/350769/2 "2024-01-11T20:13:38Z")

</div>

Hi, welcome to the community @wrender1.

There are a few similar forum posts on this subject that might be helpful to look at here:

- [Extracting the JSON fields from the message - #2 by Raed](https://discuss.elastic.co/t/extracting-the-json-fields-from-the-message/226590/2)
- [Parse json in Log field to get individual fields for visualization - #7 by aaron-nimocks](https://discuss.elastic.co/t/parse-json-in-log-field-to-get-individual-fields-for-visualization/291244/7)
- [Extracting some JSON fields from the message - #7 by Badger](https://discuss.elastic.co/t/extracting-some-json-fields-from-the-message/296479/7)

Best,  
Jessica

---

<div class="post-metadata">

**Author:** ![wrender1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wrender1/32/130685_2.png) [@wrender1](https://discuss.elastic.co/u/wrender1)\
**Post date:** [January 11, 2024, 8:37pm UTC](https://discuss.elastic.co/t/how-to-parse-json-from-message-field/350769/3 "2024-01-11T20:37:55Z")

</div>

Thanks for the links. Since I was using Elastic Agent, I was able to do it with this processor:

```auto
processors:
  - if:
      regexp:
        message: "^.*APP-METRIC.*$"     
    then:
    # Rename fields entry as it seems to conflict with elastic field names
    - replace:
        fields:
          - field: "message"
            pattern: "fields"
            replacement: "myappname"
        ignore_missing: true
        fail_on_error: false
    # Strip out anything in the message field left of APP-METRIC as it is not wanted
    - replace:
        fields:
          - field: "message"
            pattern: "^.*APP-METRIC"
            replacement: ""
        ignore_missing: true
        fail_on_error: false
    # Decode the remaining message field to json
    - decode_json_fields:
        fields: ["message"]
        process_array: true
        max_depth: 1
        target: "myappstats"
        overwrite_keys: false
        add_error_key: false

```

---

<div class="post-metadata">

**Author:** ![jessgarson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jessgarson/32/129841_2.png) [@jessgarson](https://discuss.elastic.co/u/jessgarson)\
**Post date:** [January 11, 2024, 9:59pm UTC](https://discuss.elastic.co/t/how-to-parse-json-from-message-field/350769/4 "2024-01-11T21:59:07Z")

</div>

Thanks for sharing your solution, @wrender1.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 8, 2024, 11:59pm UTC](https://discuss.elastic.co/t/how-to-parse-json-from-message-field/350769/5 "2024-02-08T23:59:08Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.