Logstash CEF codec and ECS cannot parse 'rt' field throws an error

Elastic Stack Logstash
2

A simple configuration to reproduce this is

output { stdout { codec => rubydebug { metadata => false } } }
input {
    generator {
        count => 1
        lines => [
            '2022-07-26T12:03:33+08:00 InsightCef CEF:0|CMERP|INSIGHT|1|1075168|Common Name Without Domain Name|High|rt=2022-07-26T12:03:33+08:00',
            '2022-07-26T12:03:33+08:00 InsightCef CEF:0|CMERP|INSIGHT|1|1075168|Common Name Without Domain Name|High|smac=50:3e:aa:e4:19:ae'
        ]
        codec => cef
    }
}

with --pipeline.ecs_compatibility disabled both lines are parsed, but with the default ecs_compatability setting the first one gets that timestamp parsing error. The problem is that the codec tries to normalize the field as a timestamp (you would have the same problem for end and other timestamp fields). The timestamp normalizer is looking for something that matches

"MMM dd[ yyyy] HH:mm:ss[.SSSSSSSSS][.SSSSSS][.SSS][ zzz]"

From what I have read, the rt field should be formatted as "MMM dd yyyy HH:mm:ss or milliseconds since epoch (Jan 1st 1970)". It does not accept ISO8601 style formats. The parser expects a month name at the start of the timestamp string so when it gets a year it "could not be parsed at index 0".

Personally I think demanding a specific format for rt/end etc is A Bridge Too Far, but that's not my call.

To fix this, stop using a cef codec on the initial input. grok the rt= out of the message and parse it using a date filter, then mutate+gsub it out of the message. Then use a tcp output to send the event to a tcp input with a cef codec. An example of that is here. Do not try pipeline-to-pipeline communication, that ignores the codec setting.