# Multi line pattern for different services to one logstash **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [October 24, 2018, 4:38pm UTC](https://discuss.elastic.co/t/multi-line-pattern-for-different-services-to-one-logstash/153848 "2018-10-24T16:38:51Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![kuna](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kuna/32/99295_2.png) [@kuna](https://discuss.elastic.co/u/kuna)\ **Post date:** [October 24, 2018, 4:38pm UTC](https://discuss.elastic.co/t/multi-line-pattern-for-different-services-to-one-logstash/153848/1 "2018-10-24T16:38:51Z")
I have installed file beat agents for different service and pointed them to Logstash and applied my filter for it but noticed multiple logs are coming in a single log even though i was using multi line pattern to differentiate my logs based on Timestamp. **sample log** : 2018-10-24 11:39:50.815 [http-nio-8080-exec-9] [XXXX] [supportops@XXXX.com] INFO c.s.m.c.BusinessClassCodeController - getBusinessClassCodes :Retrieving Business Class Code by CodeType :business-class-one and size :: 1 **filebeat configuration** multiline.pattern: '[0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2}' multiline.negate: true multiline.match: after **Grok filter** input { beats { port =\> 5044 } } filter { grok { ``` match => ["message","(?%{YEAR}[/-]%{MONTHNUM}[/-]%{MONTHDAY}[/]%{HOUR}:%{MINUTE}(?::%{SECOND})(?![0-9]))\s+(\[%{USERNAME:thread}\])\s+(\[%{USERNAME:tenentId}\])\s+(\[(?%{WORD}+@[0-9A-Za-z][0-9A-Za-z-]{0,62}+\w+.[a-z]{0,62})\]) %{WORD:loglevel} %{JAVACLASS:classname} %{GREEDYDATA:logmessage}"] match => ["message","(?%{YEAR}[/-]%{MONTHNUM}[/-]%{MONTHDAY}%{SPACE}%{HOUR}:%{MINUTE}(?::%{SECOND})(?![0-9]))\s+(\[%{USERNAME:thread}\])\s+(\[%{SPACE}])\s+(\[%{SPACE}])%{SPACE}%{WORD:loglevel}%{SPACE}%{JAVACLASS:classname}%{SPACE}[/-]%{GREEDYDATA:logmessage}"] ``` } mutate { remove\_field =\> ["[beat][name]", "[host][name]", "[beat][version]", "offset","[input][type]"] } } output { elasticsearch { hosts =\> ["localhost:9200"] } stdout { codec =\> rubydebug } } Filter is applied successfully but i am seeing multiple logs i.e next logs till an extent are attached as greedy data as below. **Output from kibana only _logmessage_ is given hear** logmessage : - getDayparts :Retrieving all DayPart :: 13 2018-10-24 11:39:50.815 [http-nio-8080-exec-9] [XXXX] [supportops@XXXX.com] INFO c.s.m.c.BusinessClassCodeController - getBusinessClassCodes :Retrieving Business Class Code by CodeType :business-class-one and size :: 1 2018-10-24 11:39:51.010 [http-nio-8080-exec-1] [XXXX] [supportops@XXXX.com] INFO c.s.m.c.BusinessClassCodeController - getBusinessClassCodes :Retrieving Business Class Code by CodeType :business-class-three and size :: 1 ---
**Author:** ![pierhugues](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierhugues/32/48383_2.png) [@pierhugues](https://discuss.elastic.co/u/pierhugues)\ **Post date:** [October 25, 2018, 1:40pm UTC](https://discuss.elastic.co/t/multi-line-pattern-for-different-services-to-one-logstash/153848/2 "2018-10-25T13:40:40Z")
@kuna If I understand correctly the multiline is not correctly applied to the events and you get two different event in a single event? ---
**Author:** ![kuna](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kuna/32/99295_2.png) [@kuna](https://discuss.elastic.co/u/kuna)\ **Post date:** [October 26, 2018, 4:17am UTC](https://discuss.elastic.co/t/multi-line-pattern-for-different-services-to-one-logstash/153848/3 "2018-10-26T04:17:35Z")
@pierhugues might be correct!! BTW for java stack trace it is working fine.. their i don't see any issue.. ---
**Author:** ![pierhugues](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierhugues/32/48383_2.png) [@pierhugues](https://discuss.elastic.co/u/pierhugues)\ **Post date:** [October 26, 2018, 12:16pm UTC](https://discuss.elastic.co/t/multi-line-pattern-for-different-services-to-one-logstash/153848/4 "2018-10-26T12:16:14Z")
One thing that strikes me is the pattern that you are defining is not anchored regular expression. ```auto '^[0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2}' ``` Adding ^ at the beginning force it to look up the pattern at the beginninge of the string. ---
**Author:** ![kuna](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kuna/32/99295_2.png) [@kuna](https://discuss.elastic.co/u/kuna)\ **Post date:** [October 29, 2018, 6:23am UTC](https://discuss.elastic.co/t/multi-line-pattern-for-different-services-to-one-logstash/153848/5 "2018-10-29T06:23:00Z")
@pierhugues i used the same pattern but still its not working .. ---
**Author:** ![Andrew\_Cholakian1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrew_cholakian1/32/3612_2.png) [@Andrew\_Cholakian1](https://discuss.elastic.co/u/Andrew_Cholakian1)\ **Post date:** [October 29, 2018, 7:16pm UTC](https://discuss.elastic.co/t/multi-line-pattern-for-different-services-to-one-logstash/153848/6 "2018-10-29T19:16:12Z")
@kuna can you create a minimal test case + config? In this case that would be a fake log file with multiple entries that get incorrectly interleaved by filebeat. That would be helpful in debugging this. ---
**Author:** ![pierhugues](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierhugues/32/48383_2.png) [@pierhugues](https://discuss.elastic.co/u/pierhugues)\ **Post date:** [October 31, 2018, 1:29pm UTC](https://discuss.elastic.co/t/multi-line-pattern-for-different-services-to-one-logstash/153848/7 "2018-10-31T13:29:30Z")
@kuna You could also use [https://github.com/hartfordfive/filebeat-multiline-tester](https://github.com/hartfordfive/filebeat-multiline-tester) or this playground [https://play.golang.org/p/uAd5XHxscu](https://play.golang.org/p/uAd5XHxscu) to test your patterns. ---
**Author:** ![kuna](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kuna/32/99295_2.png) [@kuna](https://discuss.elastic.co/u/kuna)\ **Post date:** [November 2, 2018, 5:39am UTC](https://discuss.elastic.co/t/multi-line-pattern-for-different-services-to-one-logstash/153848/8 "2018-11-02T05:39:18Z")
@Andrew_Cholakian1 when i tried running one application and used beats to push the logs their i dint see any issue but when i run multiple services using multiple beats its not working in the same way.. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 30, 2018, 5:39am UTC](https://discuss.elastic.co/t/multi-line-pattern-for-different-services-to-one-logstash/153848/9 "2018-11-30T05:39:18Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.