# Multiline syslogs group with the same PID? **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [May 18, 2016, 10:04pm UTC](https://discuss.elastic.co/t/multiline-syslogs-group-with-the-same-pid/50379 "2016-05-18T22:04:02Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Shiekron](https://avatars.discourse-cdn.com/v4/letter/s/b19c9b/32.png) [@Shiekron](https://discuss.elastic.co/u/Shiekron)\ **Post date:** [May 18, 2016, 10:04pm UTC](https://discuss.elastic.co/t/multiline-syslogs-group-with-the-same-pid/50379/1 "2016-05-18T22:04:02Z")
I'm curious on how would you attempt to create a multiline match on just the PID to put it on a single event? I'm currently using this: > prospectors: > - > paths: > - /usr/local/log/opensips.log > encoding: plain > input\_type: log > exclude\_lines: ["ACD DEBUG1 ", "isflagset is not set to"] > multiline: > pattern: '----- Received +|----- out$' > negate: true > match: after > max\_lines: 500 > timeout: 5s > tail\_files: true But there are times we have log lines that don't always follow sequentially so the logs don't group together like the following example: > May 15 19:41:45 azsbc1 /usr/local/opensips\_sbc/sbin/opensips[25432]: ----- Received SUBSCRIBE from 96.88.101.105 to sip:111@511132.retracted.com:5060 [373daac320b557f@192.168.1.113](mailto:373daac320b557f@192.168.1.113), \<179c730894a14c8\> sip:111@511132.retracted.com:5060 > May 15 19:41:45 azsbc1 /usr/local/opensips\_sbc/sbin/opensips[25401]: ----- Received SUBSCRIBE from 96.88.101.105 to sip:115@511132.retracted.com:5060 [e07c9141e7deba4@192.168.1.113](mailto:e07c9141e7deba4@192.168.1.113), sip:115@511132.retracted.com:5060 > May 15 19:41:45 azsbc1 /usr/local/opensips\_sbc/sbin/opensips[25432]: xXx = method is REGISTER or SUBSCRIBE > May 15 19:41:45 azsbc1 /usr/local/opensips\_sbc/sbin/opensips[25432]: ----- out to sip:111@511132.retracted.com:5060 / sip:10.15.7.23:5090 > May 15 19:41:45 azsbc1 /usr/local/opensips\_sbc/sbin/opensips[25401]: xXx = method is REGISTER or SUBSCRIBE > May 15 19:41:45 azsbc1 /usr/local/opensips\_sbc/sbin/opensips[25401]: ----- out to sip:115@511132.retracted.com:5060 / sip:10.15.7.35:5090 > May 15 19:41:45 azsbc1 /usr/local/opensips\_sbc/sbin/opensips[25431]: ----- Received SUBSCRIBE from 96.88.101.105 to sip:118@511132.retracted.com:5060 [306cc35b3d3940d@192.168.1.113](mailto:306cc35b3d3940d@192.168.1.113), sip:118@511132.retracted.com:5060 > May 15 19:41:45 azsbc1 /usr/local/opensips\_sbc/sbin/opensips[25431]: xXx = method is REGISTER or SUBSCRIBE > May 15 19:41:45 azsbc1 /usr/local/opensips\_sbc/sbin/opensips[25431]: ----- out to sip:118@511132.retracted.com:5060 / sip:10.15.7.34:5090 > May 15 19:41:45 azsbc1 /usr/local/opensips\_sbc/sbin/opensips[25419]: ----- Received NOTIFY from 10.15.7.58 to [sip:61e05271773d14178601496e61cb7db0@10.15.7.30](mailto:sip:61e05271773d14178601496e61cb7db0@10.15.7.30) \<926cfb17-95b2-1234-699b-00219bfc76bb\>, sip:113@10.15.7.58 > May 15 19:41:45 azsbc1 /usr/local/opensips\_sbc/sbin/opensips[25419]: ---call to [sip:61e05271773d14178601496e61cb7db0@10.15.7.30](mailto:sip:61e05271773d14178601496e61cb7db0@10.15.7.30), from 10.15.7.58/5090 So I was thinking it may be best to group based on the PID, but I tried: > filebeat: > prospectors: > - > paths: > - /usr/local/log/opensips.log > encoding: plain > input\_type: log > exclude\_lines: ["ACD DEBUG1 ", "isflagset is not set to"] > multiline: > pattern: '[[[:digit:]]{5}]: +|[[[:digit:]]{5}]: ----- out$' > negate: true > match: after > max\_lines: 500 > timeout: 10s > tail\_files: true But I couldn't figure out how to make it matching based on the the PID, I am not too sure how to use (re) within the above, as I believe the other type of grouping (?s) or (?m) is unsupported according to [http://elasticsearch.qiniudn.com/guide/en/beats/filebeat/current/regexp-support.html](http://elasticsearch.qiniudn.com/guide/en/beats/filebeat/current/regexp-support.html) ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [May 19, 2016, 9:38am UTC](https://discuss.elastic.co/t/multiline-syslogs-group-with-the-same-pid/50379/2 "2016-05-19T09:38:58Z")
You are asking for correlation? Multiline is about parsing multiple consecutive lines into one event. ---
**Author:** ![Shiekron](https://avatars.discourse-cdn.com/v4/letter/s/b19c9b/32.png) [@Shiekron](https://discuss.elastic.co/u/Shiekron)\ **Post date:** [May 19, 2016, 4:13pm UTC](https://discuss.elastic.co/t/multiline-syslogs-group-with-the-same-pid/50379/3 "2016-05-19T16:13:32Z")
Yeah, I do know that, but is there a way filebeat can group events like in the above with the same PID into one? Like sometimes you see different processes writing to the log at the same time and they sometimes run over each other as you can see above, can filebeat somehow group the events with the match of the same PID until its timeout to finally group that as one event ? ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [May 19, 2016, 5:05pm UTC](https://discuss.elastic.co/t/multiline-syslogs-group-with-the-same-pid/50379/4 "2016-05-19T17:05:39Z")
No, filebeat is not parsing actual content. Regular expressions are only used for matching lines according to some pattern. You're basically asking for event correlation, which is not supported by filebeat. filebeat is about shipping logs, not processing content. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 9:51pm UTC](https://discuss.elastic.co/t/multiline-syslogs-group-with-the-same-pid/50379/5 "2017-07-05T21:51:44Z")