# How to parse stringified json in logstash

**URL:** https://discuss.elastic.co/t/how-to-parse-stringified-json-in-logstash/367254
**Category:** Logstash
**Created:** [September 27, 2024, 4:00pm UTC](https://discuss.elastic.co/t/how-to-parse-stringified-json-in-logstash/367254 "2024-09-27T16:00:25Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![vijay117](https://avatars.discourse-cdn.com/v4/letter/v/e0b2c6/32.png) [@vijay117](https://discuss.elastic.co/u/vijay117)
#### Post date: [September 27, 2024, 4:00pm UTC](https://discuss.elastic.co/t/how-to-parse-stringified-json-in-logstash/367254/1 "2024-09-27T16:00:25Z")

</div>

```auto
{
    "name": "cmg-notification-service App",
    "hostname": "notify-service-main-8cbfffc56-dq924",
    "pid": 1,
    "crn": "XXXXXXXXXXX",
    "url": "/v2/notify/sms",
    "requestId": "IJaxVCtAilpjSRODkySgaQ==",
    "level": 30,
    "msg": "Kafka payload for PersonsNotificationIssued {\n metadata: {\n version_number: '1.0.0',\n lob_schema: '2012',\n host_name: 'notifyservice.XXX.XXX-XXX.XXX-eks.XXXcloud.uk',\n ip_address: 'notifyservice.cms.XXX-dev.XXX-eks.XXXcloud.uk',\n sor: 'CMG Notification Service',\n uri: 'https://notifyservice.cms.XXX-dev.XXX-eks.XXXcloud.uk/v2/notify',\n transaction_parent_id: 'c4cd7290-6452-4cfe-9531-da79b190d8cd',\n transaction_child_id: '756XXXXXXX',\n transaction_date: '2024-09-27T08:24:36.312Z'\n },\n business_interest: {\n customer_ref_number: '{\"DEK\":\"NcGRIjKZPckaf5f+trwO5s/3uDTIB6YC6dfIklFXbWr/e+4OBJAv/HEIdI2L++o6xmvG3Z0KD8Q8bm/iiwIyRw==\",\"message\":\"NcGRIjKZPckaf5f+trwO5mqo5BH1Xs8UeAv/7OttQK7hX5/YnAlT/n/mYkQ=\",\"crypto_metadata\":{\"group\":\"CentralGovRef\",\"version\":\"1\"}}',\n transaction_ref_number: '{\"DEK\":\"RskioQAlkcZXC3burgPxHlahWpz7ip7pmLcW1h5QugTBKcGH4EDeLObrYidR25KhoSSj4Df69xLASlU4BekcPg==\",\"message\":\"RskioQAlkcZXC3burgPxHjtfjp/izwKpsspu6e0WXAJ74O6pgJpdeNCrmwY1\",\"crypto_metadata\":{\"group\":\"CentralGovRef\",\"version\":\"1\"}}',\n notification_ref_number: 'c4cd7290-6452-4cfe-9531-da79b190d8cd',\n notification_channel: 'SMS',\n templateId: 'S0001',\n templateVersion: 1,\n emailNotificationPreferred: true,\n source: 'XXXXX XXXXX XXXXX',\n failure_reason: undefined,\n AdditionalInformation: { transactionRef: '7XXXXXXXX', amount: '5.45' }\n }\n}",
    "time": "2024-09-27T08:24:36.314Z",
    "v": 0
}

This is a log (message)i have in cloud watch and i want to parse using json.
when i used "json {source => message} " it parses some which are in JSON format except "msg".
when i use json {source => msg} to further parse it throws me an error _jsonparsefailure

I came to know that this is in stringified json, how to parse this ?

```

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [September 27, 2024, 8:22pm UTC](https://discuss.elastic.co/t/how-to-parse-stringified-json-in-logstash/367254/2 "2024-09-27T20:22:18Z")

</div>

> [@vijay117](#):
>
> `I came to know that this is in stringified json, how to parse this ?`

No, it is not stringified JSON, although it does have stringified JSON embedded in it (the two ref\_number fields). You can try parsing it with a kv filter

```
    json { source => "message" remove_field => ["message"] }

    kv {
        source => "msg"
        field_split_pattern => "\\n" # Or perhaps "\n"
        value_split_pattern => ": "
        trim_key => " "
        trim_value => ",'"
        remove_field => ["msg"]
    }
    json { source => "transaction_ref_number" target => "transaction_ref_number" }
    json { source => "customer_ref_number" target => "customer_ref_number" }

```

Note that the structure of fields within the business\_interest and metadata objects is lost. Everything gets flattened. If that is a problem then another approach would be

```
    json { source => "message" remove_field => ["message"] }

    grok {
        break_on_match => false
        match => {
            msg => [
                "metadata: %{GREEDYDATA:[@metadata][metadata]}},",
                "business_interest: %{GREEDYDATA:[@metadata][bi]}\n }"
            ]
        }
        #remove_field => ["msg"]
    }
    kv {
        source => "[@metadata][metadata]"
        target => "metadata"
        field_split_pattern => "\\n"
        value_split_pattern => ": "
        trim_key => " "
        trim_value => ",'"
    }
    kv {
        source => "[@metadata][bi]"
        target => "business_interest"
        field_split_pattern => "\\n"
        value_split_pattern => ": "
        trim_key => " "
        trim_value => ",'"
    }
    json { source => "[business_interest][transaction_ref_number]" target => "[business_interest][transaction_ref_number]" }
    json { source => "[business_interest][customer_ref_number]" target => "[business_interest][customer_ref_number]" }

```

but those grok patterns will be fragile against changes in the message format.

---

<div class="post-metadata">

### Author: ![vijay117](https://avatars.discourse-cdn.com/v4/letter/v/e0b2c6/32.png) [@vijay117](https://discuss.elastic.co/u/vijay117)
#### Post date: [September 29, 2024, 8:59am UTC](https://discuss.elastic.co/t/how-to-parse-stringified-json-in-logstash/367254/3 "2024-09-29T08:59:34Z")

</div>

Thanks @Badger I will try the above, and by the way what is kv filter? (I am new to ELK)

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [September 29, 2024, 2:12pm UTC](https://discuss.elastic.co/t/how-to-parse-stringified-json-in-logstash/367254/4 "2024-09-29T14:12:20Z")

</div>

It parses key/value pairs. The documentation is [here](https://www.elastic.co/guide/en/logstash/current/plugins-filters-kv.html).
