# Is Logstash able to inline correct invalid JSON?

**URL:** <https://discuss.elastic.co/t/is-logstash-able-to-inline-correct-invalid-json/378473>\
**Category:** Logstash\
**Created:** [May 23, 2025, 3:29pm UTC](https://discuss.elastic.co/t/is-logstash-able-to-inline-correct-invalid-json/378473 "2025-05-23T15:29:16Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Marcos\_Mondragon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marcos_mondragon/32/105509_2.png) [@Marcos\_Mondragon](https://discuss.elastic.co/u/Marcos_Mondragon)\
**Post date:** [May 23, 2025, 3:29pm UTC](https://discuss.elastic.co/t/is-logstash-able-to-inline-correct-invalid-json/378473/1 "2025-05-23T15:29:16Z")

</div>

I'm trying to determine if there is a way within a Logstash conf file to correct an invalid JSON field being passed in? Here is the data:

**{ "trace":**"ID-in":"83455E37DD688327", "ID-out":"83455E37DD688327", "reply-to-in":"l.metx.metx+1003.cosma", "route-in":"b.metx.cosma", "route-out":"b.metx.cosma", "time-in":"1747938599239", "time-out":"1747938599239", "class-in":"0", "class-out":"0", "from":"TQ.wifi", "to":"TNB.Inbound.Ex", "latency":"8", "handling-time":"0", "transport-used":"wifi", "sh-code-in":"0", "sh-code-out":"0", "emp-msg-id":"6010" }

The 3rd party application is also sending out correct data:

**{ "name":"trace",**"ID-in":"50AF7506625A625", "ID-out":"294E131FAB4A5684", "crc-out":"64657", "fpid":"-", "reply-to-in":"l.metx.metx+1003.cosma", "route-in":"b.metx.cosma", "route-out":"b.metx.cosma", "time-in":"1747938669306", "time-out":"1747938669306", "size-in":"157", "size-out":"118", "class-in":"0", "class-out":"0", "from":"metx.l.metx.1003.CosmaApp.FromApplication", "to":"RBOutbound.Ex", "latency":"0", "handling":"0", "transport-used":"-", "emp-msg-id":"6010" }

Which I am able to parse like so:

```auto
grok {
      match => {
        "message" => '%{SYSLOGTIMESTAMP:tmc_timestamp} %{SYSLOGHOST:tmc_hostname} %{DATA:tmc_program}\[%{POSINT:tmc_app_pid}\]: .*? %{TIMESTAMP_ISO8601:ruby_timestamp} %{DATA:tmc_unit} %{DATA:tmc_svcname} .*? \[.*?="%{IP:itcm_major}%{DATA:itcm_minor}\"\].*?%{GREEDYDATA:tmc_json}'
      }
    }

```

Right now, I'm tossing away this invalid data due to this invalid section: **"trace":**. I'd like to change it to: **"name":"trace",**. I would be greatly appreciative of any suggestions on this dilemma.

Regards,

Marc M.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 23, 2025, 3:43pm UTC](https://discuss.elastic.co/t/is-logstash-able-to-inline-correct-invalid-json/378473/2 "2025-05-23T15:43:17Z")

</div>

You could use mutate+gsub and then a json filter

```
mutate { gsub => ["message", '{ "trace":', '{ "name": "trace",'] }

```

---

<div class="post-metadata">

**Author:** ![Marcos\_Mondragon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marcos_mondragon/32/105509_2.png) [@Marcos\_Mondragon](https://discuss.elastic.co/u/Marcos_Mondragon)\
**Post date:** [May 23, 2025, 4:24pm UTC](https://discuss.elastic.co/t/is-logstash-able-to-inline-correct-invalid-json/378473/3 "2025-05-23T16:24:30Z")

</div>

I had it in my mind that that might be the way but didn't have the requisite YAML skills to make it work. I used your example:

```auto
    mutate { gsub => ["message", '{ "trace":', '{ "name": "trace",'] }

    grok {
      match => {
        "message" => '%{SYSLOGTIMESTAMP:tmc_timestamp} %{SYSLOGHOST:tmc_hostname} %{DATA:tmc_program}\[%{POSINT:tmc_app_pid}\]: .*? %{TIMESTAMP_ISO8601:ruby_timestamp} %{DATA:tmc_unit} %{DATA:tmc_svcname} .*? \[.*?="%{IP:itcm_major}%{DATA:itcm_minor}\"\].*?%{GREEDYDATA:tmc_json}'
      }
    }

```

But it came back with an error I've seen in my mutate attempts:

**May 23 11:18:27 lab-log-01.ptclab.local logstash[11747]: [2025-05-23T11:18:27,764][WARN][logstash.filters.json][main][ce1d02240146164699a81143de69b790a4dd2d7b9e57f39ae8a7bb7816e034b4] Error parsing json {:source=\>"tmc\_json", :raw=\>" { "trace": "ID-in":"25D700D48FC600BE", "ID-out":"-", "reply-to-in":"-", "route-in":"U", "route-out":"-", "time-in":"1748017105575", "time-out":"1748017105576", "class-in":"-", "class-out":"-", "from":"MR\_Command\_Queue\_9cb48acd-3806-4a2b-a09a-8c04d805d598", "to":"-", "latency":"-", "handling-time":"1", "transport-used":"-", "sh-code-in":"0", "sh-code-out":"-", "emp-msg-id":"-" }", :exception=\>#\<LogStash::Json::ParserError: Unexpected character (':' (code 58)): was expecting comma to separate Object entries**

I've tried and failed to make that character conversion from ':' to ',' in my attempts.

---

<div class="post-metadata">

**Author:** ![Marcos\_Mondragon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marcos_mondragon/32/105509_2.png) [@Marcos\_Mondragon](https://discuss.elastic.co/u/Marcos_Mondragon)\
**Post date:** [May 23, 2025, 4:43pm UTC](https://discuss.elastic.co/t/is-logstash-able-to-inline-correct-invalid-json/378473/4 "2025-05-23T16:43:44Z")

</div>

Badger's solution in fact did work. I went back into my filter and removed the if else block and it started working. This is the now simplified version of my conf file:

```auto
# TMC event filter
##
filter {
  if ( "Slot10" in [environment] ) {

    mutate { add_tag => "TMC" }

    mutate { gsub => ["message", '{ "trace":', '{ "name": "trace",'] }

    grok {
      match => {
        "message" => '%{SYSLOGTIMESTAMP:tmc_timestamp} %{SYSLOGHOST:tmc_hostname} %{DATA:tmc_program}\[%{POSINT:tmc_app_pid}\]: .*? %{TIMESTAMP_ISO8601:ruby_timestamp} %{DATA:tmc_unit} %{DATA:tmc_svcname} .*? \[.*?="%{IP:itcm_major}%{DATA:itcm_minor}\"\].*?%{GREEDYDATA:tmc_json}'
      }
    }

    mutate { add_field => { "itcm_version" => "%{[itcm_major]}%{[itcm_minor]}" } }

    json {
      source => "tmc_json"
      target => "tmc"
      tag_on_failure => ["_jsonparsefailure"]
    }

    if "_jsonparsefailure" in [tags] {
      drop { }
    }

  }
}

```

Thank you Badger!