# How to match lines in unstructured log starting with specific string

**URL:** <https://discuss.elastic.co/t/how-to-match-lines-in-unstructured-log-starting-with-specific-string/284112>\
**Category:** Logstash\
**Created:** [September 13, 2021, 5:50pm UTC](https://discuss.elastic.co/t/how-to-match-lines-in-unstructured-log-starting-with-specific-string/284112 "2021-09-13T17:50:32Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [September 13, 2021, 6:38pm UTC](https://discuss.elastic.co/t/how-to-match-lines-in-unstructured-log-starting-with-specific-string/284112/2 "2021-09-13T18:38:02Z")

</div>

If you want to combine all the lines from one file into a single document then you could do it using an aggregate filter, but I would use a multiline codec to read the entire file as a single event as described [here](https://discuss.elastic.co/t/parsing-array-of-json-objects-with-logstash-and-injesting-to-elastic/203197/2).

I would then do the processing with a ruby filter.

```
    ruby {
        code => '
            lines = event.get("message").lines(chomp: true)
            newMessage = ""
            theRest = ""
            lines.each { |x|
                if x =~ /^(ABC|CDE)/
                    newMessage += x + ","
                else
                    unless x =~ /^(\s|Timing)/
                        theRest += x + ","
                    end
                end
            }
            event.set("message", newMessage)
            event.set("data", theRest)
        '
    }

```

Obviously you will want to tune those regular expressions. With the file you showed that will get you

```
      "data" => "Quilting with 1 groups of 0 I/O tasks.,DYNAMICS OPTION: Eulerian Mass Coordinate,",
   "message" => "ABC V4.1.2 MODEL,ABC restart, LBC starts at 1979-12-19_00:00:00 and restart starts at 1979-12-19_00:00:00,CDE: 0 hostname: xxxx.xxx,"
```

---

_[View the full topic](https://discuss.elastic.co/t/how-to-match-lines-in-unstructured-log-starting-with-specific-string/284112)._