Is Logstash able to inline correct invalid JSON?

Elastic Stack Logstash
1

I'm trying to determine if there is a way within a Logstash conf file to correct an invalid JSON field being passed in? Here is the data:

{ "trace": "ID-in":"83455E37DD688327", "ID-out":"83455E37DD688327", "reply-to-in":"l.metx.metx+1003.cosma", "route-in":"b.metx.cosma", "route-out":"b.metx.cosma", "time-in":"1747938599239", "time-out":"1747938599239", "class-in":"0", "class-out":"0", "from":"TQ.wifi", "to":"TNB.Inbound.Ex", "latency":"8", "handling-time":"0", "transport-used":"wifi", "sh-code-in":"0", "sh-code-out":"0", "emp-msg-id":"6010" }

The 3rd party application is also sending out correct data:

{ "name":"trace", "ID-in":"50AF7506625A625", "ID-out":"294E131FAB4A5684", "crc-out":"64657", "fpid":"-", "reply-to-in":"l.metx.metx+1003.cosma", "route-in":"b.metx.cosma", "route-out":"b.metx.cosma", "time-in":"1747938669306", "time-out":"1747938669306", "size-in":"157", "size-out":"118", "class-in":"0", "class-out":"0", "from":"metx.l.metx.1003.CosmaApp.FromApplication", "to":"RBOutbound.Ex", "latency":"0", "handling":"0", "transport-used":"-", "emp-msg-id":"6010" }

Which I am able to parse like so:

grok {
      match => {
        "message" => '%{SYSLOGTIMESTAMP:tmc_timestamp} %{SYSLOGHOST:tmc_hostname} %{DATA:tmc_program}\[%{POSINT:tmc_app_pid}\]: .*? %{TIMESTAMP_ISO8601:ruby_timestamp} %{DATA:tmc_unit} %{DATA:tmc_svcname} .*? \[.*?="%{IP:itcm_major}%{DATA:itcm_minor}\"\].*?%{GREEDYDATA:tmc_json}'
      }
    }

Right now, I'm tossing away this invalid data due to this invalid section: "trace":. I'd like to change it to: "name":"trace",. I would be greatly appreciative of any suggestions on this dilemma.

Regards,

Marc M.

2

You could use mutate+gsub and then a json filter

mutate { gsub => [ "message", '{ "trace":', '{ "name": "trace",' ] }
1 Like
3

I had it in my mind that that might be the way but didn't have the requisite YAML skills to make it work. I used your example:

    mutate { gsub => [ "message", '{ "trace":', '{ "name": "trace",' ] }

    grok {
      match => {
        "message" => '%{SYSLOGTIMESTAMP:tmc_timestamp} %{SYSLOGHOST:tmc_hostname} %{DATA:tmc_program}\[%{POSINT:tmc_app_pid}\]: .*? %{TIMESTAMP_ISO8601:ruby_timestamp} %{DATA:tmc_unit} %{DATA:tmc_svcname} .*? \[.*?="%{IP:itcm_major}%{DATA:itcm_minor}\"\].*?%{GREEDYDATA:tmc_json}'
      }
    }

But it came back with an error I've seen in my mutate attempts:

May 23 11:18:27 lab-log-01.ptclab.local logstash[11747]: [2025-05-23T11:18:27,764][WARN ][logstash.filters.json ][main][ce1d02240146164699a81143de69b790a4dd2d7b9e57f39ae8a7bb7816e034b4] Error parsing json {:source=>"tmc_json", :raw=>" { "trace": "ID-in":"25D700D48FC600BE", "ID-out":"-", "reply-to-in":"-", "route-in":"U", "route-out":"-", "time-in":"1748017105575", "time-out":"1748017105576", "class-in":"-", "class-out":"-", "from":"MR_Command_Queue_9cb48acd-3806-4a2b-a09a-8c04d805d598", "to":"-", "latency":"-", "handling-time":"1", "transport-used":"-", "sh-code-in":"0", "sh-code-out":"-", "emp-msg-id":"-" }", :exception=>#<LogStash::Json::ParserError: Unexpected character (':' (code 58)): was expecting comma to separate Object entries

I've tried and failed to make that character conversion from ':' to ',' in my attempts.

4

Badger's solution in fact did work. I went back into my filter and removed the if else block and it started working. This is the now simplified version of my conf file:

# TMC event filter
##
filter {
  if ( "Slot10" in [environment] ) {

    mutate { add_tag => "TMC" }

    mutate { gsub => [ "message", '{ "trace":', '{ "name": "trace",' ] }

    grok {
      match => {
        "message" => '%{SYSLOGTIMESTAMP:tmc_timestamp} %{SYSLOGHOST:tmc_hostname} %{DATA:tmc_program}\[%{POSINT:tmc_app_pid}\]: .*? %{TIMESTAMP_ISO8601:ruby_timestamp} %{DATA:tmc_unit} %{DATA:tmc_svcname} .*? \[.*?="%{IP:itcm_major}%{DATA:itcm_minor}\"\].*?%{GREEDYDATA:tmc_json}'
      }
    }

    mutate { add_field => { "itcm_version" => "%{[itcm_major]}%{[itcm_minor]}" } }

    json {
      source => "tmc_json"
      target => "tmc"
      tag_on_failure => ["_jsonparsefailure"]
    }

    if "_jsonparsefailure" in [tags] {
      drop { }
    }

  }
}

Thank you Badger!

1 Like