If you want to combine all the lines from one file into a single document then you could do it using an aggregate filter, but I would use a multiline codec to read the entire file as a single event as described here.
I would then do the processing with a ruby filter.
ruby {
code => '
lines = event.get("message").lines(chomp: true)
newMessage = ""
theRest = ""
lines.each { |x|
if x =~ /^(ABC|CDE)/
newMessage += x + ","
else
unless x =~ /^(\s|Timing)/
theRest += x + ","
end
end
}
event.set("message", newMessage)
event.set("data", theRest)
'
}
Obviously you will want to tune those regular expressions. With the file you showed that will get you
"data" => "Quilting with 1 groups of 0 I/O tasks.,DYNAMICS OPTION: Eulerian Mass Coordinate,",
"message" => "ABC V4.1.2 MODEL,ABC restart, LBC starts at 1979-12-19_00:00:00 and restart starts at 1979-12-19_00:00:00,CDE: 0 hostname: xxxx.xxx,"