Parse json in string in json (nested json)

[Romanian_Coder]

I have next json in input

 {
  "partitionId": 3,
  "value": {
    "name": "updatedPaymentInfo",
    "value": "{\"reference\":\"Z091702210000015\",\"transactionId\":\"CC11702210000020\",\"amountDTO\":{\"value\":10000,\"currency\":\"RUR\",\"minorUnits\":100},\"status\":\"V\",\"fields\":[{\"type\":\"CHAR\",\"value\":\"CC61401210000183\",\"name\":\"#HUMN\"}]}",
    "workflowKey": 2251799815337822,
    "workflowInstanceKey": 6755399442908649,
    "scopeKey": 6755399442908690
  },
  "sourceRecordPosition": 3744890,
  "valueType": "VARIABLE",
  "position": 3744892,
  "key": 6755399442908692,
  "timestamp": 1613564803494,
  "recordType": "EVENT",
  "intent": "CREATED",
  "rejectionType": "NULL_VAL",
  "rejectionReason": "",
  "brokerVersion": "0.26.0"
}

How I can parse nested json as string in top-level? I want in output get

{
  "partitionId": 3,
  "value": {
    "name": "updatedPaymentInfo",
    "workflowKey": 2251799815337822,
    "workflowInstanceKey": 6755399442908649,
    "scopeKey": 6755399442908690
  },
  "sourceRecordPosition": 3744890,
  "valueType": "VARIABLE",
  "position": 3744892,
  "key": 6755399442908692,
  "timestamp": 1613564803494,
  "recordType": "EVENT",
  "intent": "CREATED",
  "rejectionType": "NULL_VAL",
  "rejectionReason": "",
  "brokerVersion": "0.26.0",
  "variables" :  {"reference":"Z091702210000015","transactionId":"CC11702210000020"\"amountDTO":{"value":10000,"currency":"RUR","minorUnits":100},"status":"V","fields":[{"type":"CHAR","value":"CC61401210000183","name":"#HUMN"}]}
}

I am trying by this filter, but nothing succeeded

 filter {
    json {
        source => "message"
    }
    if [valueType] == "JOB_BATCH" {
        drop { }
    }
    if [valueType] == "VARIABLE" {
       mutate{
            replace => [ "value", "%{value}" ]
            gsub => [ 'value','\n','']
        }

        if [value] =~ /^{.*}$/ {
            json { source => message }
        }
    }
    mutate {
        remove_field => [ "syslog_hostname", "spanExportable", "timestamp"]
    }

}

[val (Val Crettaz)]

Logstash provides a filter called json that you can use to parse the [value][value] field:

filter {
  json {
    source => "[value][value]"
    target => "variables"
  }
}

If you don't specify any target field, the parsed data will be stored at the root level.

In your case, I think you got it right, except that the field name is not correct, it should be [value][value] instead of message

[Romanian_Coder]

Thanks! This solved my problem!

[val (Val Crettaz)]

Cool, happy to help!