# Tricky Multiline pattern/configuration **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [October 2, 2017, 3:49pm UTC](https://discuss.elastic.co/t/tricky-multiline-pattern-configuration/102450 "2017-10-02T15:49:04Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![sbuch](https://avatars.discourse-cdn.com/v4/letter/s/34f0e0/32.png) [@sbuch](https://discuss.elastic.co/u/sbuch)\ **Post date:** [October 2, 2017, 3:49pm UTC](https://discuss.elastic.co/t/tricky-multiline-pattern-configuration/102450/1 "2017-10-02T15:49:04Z")
Hi I have a very tricky situation with two types of multiline messages in one file and of course no application support to change it 🙂 ``` Example 1: 20170930 14:20:01.003 message 1 line 1 message 1 line 2 20170930 14:20:01.004 message 2 Example 2: 20170930 14:23:04.013 message 1 line 1 20170930 14:23:04.014 message 1 executed in 99ms ``` We need two different multiline patterns. One has to be negated (every line that doesn´t look like message 1 line 1 should be a multiline) and the other one should not be negated (every line that consists of "excuted in 99 ms" should be a multiline aswell). Is there a way to realize that? Easiest way would be one negate pattern and one nonegate pattern. Thanks a lot for your help! ---
**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\ **Post date:** [October 3, 2017, 1:32pm UTC](https://discuss.elastic.co/t/tricky-multiline-pattern-configuration/102450/2 "2017-10-03T13:32:10Z")
Hi! You can specify two multiline patterns by connecting them using `|` which stands for or. So you could write `'[^negated-pattern]|pattern'`. You can test you multiline regex using this Go Playground: [https://play.golang.org/p/uAd5XHxscu](https://play.golang.org/p/uAd5XHxscu) More info on how to test it: [https://www.elastic.co/guide/en/beats/filebeat/current/multiline-examples.html#\_testing\_your\_regexp\_pattern\_for\_multiline](https://www.elastic.co/guide/en/beats/filebeat/current/multiline-examples.html#_testing_your_regexp_pattern_for_multiline) Let us know, if you need further help. ---
**Author:** ![sbuch](https://avatars.discourse-cdn.com/v4/letter/s/34f0e0/32.png) [@sbuch](https://discuss.elastic.co/u/sbuch)\ **Post date:** [October 4, 2017, 11:55am UTC](https://discuss.elastic.co/t/tricky-multiline-pattern-configuration/102450/4 "2017-10-04T11:55:05Z")
> [@sbuch](#): > > 20170930 14:23:04.014 message 1 executed in 99ms Hi! Thank you for your answer! Unfortunateley I think that it is not working like this. I am using the following pattern: `^[^20[0-9]{2}(0[1-9]|1[0-2])([0-2][0-9]|3[0-1])\s([0-1][0-9]|2[0-3]):[0-5][0-9]:[0-5][0-9].[0-9]{3}\s(message)\s\d+\s(executed in)\s\d+(ms)]|^20[0-9]{2}(0[1-9]|1[0-2])([0-2][0-9]|3[0-1])\s([0-1][0-9]|2[0-3]):[0-5][0-9]:[0-5][0-9].[0-9]{3}\s(message)\s\d+\s(line)\s\d+` But it doesn´t match the following line: `20170930 14:23:04.014 message 1 executed in 99ms` It matches all the lines that do not look like this line: `20170930 14:20:01.003 message 1 line 1` Here´s is the link to the Go Playground: [https://play.golang.org/p/Vo2AJjPzaN](https://play.golang.org/p/Vo2AJjPzaN) Thanks a lot for your help! ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [October 4, 2017, 12:18pm UTC](https://discuss.elastic.co/t/tricky-multiline-pattern-configuration/102450/5 "2017-10-04T12:18:56Z")
Indeed quite tricky. The `[^...]` clause does not negate a pattern, but negates a character class. My attempt uses the pattern `(^[^\d]{8}|executed in)` with negate false. This regex checks the log line is not starting with 8 digits (the date) or does contain the substring `executed in`. See the playground: [https://play.golang.org/p/0TcDlVySS3](https://play.golang.org/p/0TcDlVySS3) ---
**Author:** ![sbuch](https://avatars.discourse-cdn.com/v4/letter/s/34f0e0/32.png) [@sbuch](https://discuss.elastic.co/u/sbuch)\ **Post date:** [October 4, 2017, 12:55pm UTC](https://discuss.elastic.co/t/tricky-multiline-pattern-configuration/102450/6 "2017-10-04T12:55:21Z")
Hi! I think I got it working! Thank you for your help 😬 ---
**Author:** ![sbuch](https://avatars.discourse-cdn.com/v4/letter/s/34f0e0/32.png) [@sbuch](https://discuss.elastic.co/u/sbuch)\ **Post date:** [October 5, 2017, 7:30am UTC](https://discuss.elastic.co/t/tricky-multiline-pattern-configuration/102450/7 "2017-10-05T07:30:21Z")
Hi again! Is it possible to use more than one negated class? ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [October 5, 2017, 9:56am UTC](https://discuss.elastic.co/t/tricky-multiline-pattern-configuration/102450/8 "2017-10-05T09:56:26Z")
You can add any character you like I think. But the more characters you add to the character class, the higher the chance of false positives. What's the issue? ---
**Author:** ![sbuch](https://avatars.discourse-cdn.com/v4/letter/s/34f0e0/32.png) [@sbuch](https://discuss.elastic.co/u/sbuch)\ **Post date:** [October 5, 2017, 10:24am UTC](https://discuss.elastic.co/t/tricky-multiline-pattern-configuration/102450/9 "2017-10-05T10:24:44Z")
For example, if I use a pattern like this(var negate = true): `(^[^\d]{3}[^\s][^\d]{3})` The following lines should be matched: > 200 300 But these are matched aswell for example: > ``` > 200 > asdf > 2603 > 1 > > ``` What is wrong with my pattern? Link to playground: [https://play.golang.org/p/9J9vZqvzQz](https://play.golang.org/p/9J9vZqvzQz) ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [October 6, 2017, 10:11am UTC](https://discuss.elastic.co/t/tricky-multiline-pattern-configuration/102450/10 "2017-10-06T10:11:59Z")
Hmm... I don't fully get it. First of all it seems you have a mix of quite some patterns in your log-file (yeah). Having all patterns in the playground in order to test and play with it, helps in creating a more robust regex (if possible). Please add a bigger corpus potential log message. Is it always 2 numbers followed of 3 digits or are there other 'similar' patterns. I kind of see what you are trying with your pattern. You try to negate a complete sequence, by negating the individual character classes. This is not possible, as one can not negate complete terms with regular expressions. Give your sample the closes I can think of is `^[\d]{7}`. But this matches any sequence of digits and spaces, taking a total of 7 bytes. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 3, 2017, 10:12am UTC](https://discuss.elastic.co/t/tricky-multiline-pattern-configuration/102450/11 "2017-11-03T10:12:05Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.