How to match lines in unstructured log starting with specific string

Elastic Stack Logstash
2

If you want to combine all the lines from one file into a single document then you could do it using an aggregate filter, but I would use a multiline codec to read the entire file as a single event as described here.

I would then do the processing with a ruby filter.

    ruby {
        code => '
            lines = event.get("message").lines(chomp: true)
            newMessage = ""
            theRest = ""
            lines.each { |x|
                if x =~ /^(ABC|CDE)/
                    newMessage += x + ","
                else
                    unless x =~ /^(\s|Timing)/
                        theRest += x + ","
                    end
                end
            }
            event.set("message", newMessage)
            event.set("data", theRest)
        '
    }

Obviously you will want to tune those regular expressions. With the file you showed that will get you

      "data" => "Quilting with   1 groups of   0 I/O tasks.,DYNAMICS OPTION: Eulerian Mass Coordinate,",
   "message" => "ABC V4.1.2 MODEL,ABC restart, LBC starts at 1979-12-19_00:00:00 and restart starts at 1979-12-19_00:00:00,CDE: 0 hostname: xxxx.xxx,"