# Filebeat multiline based on capture group **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [March 13, 2024, 12:50am UTC](https://discuss.elastic.co/t/filebeat-multiline-based-on-capture-group/355267 "2024-03-13T00:50:12Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Operate6423](https://avatars.discourse-cdn.com/v4/letter/o/3ab097/32.png) [@Operate6423](https://discuss.elastic.co/u/Operate6423)\ **Post date:** [March 13, 2024, 12:50am UTC](https://discuss.elastic.co/t/filebeat-multiline-based-on-capture-group/355267/1 "2024-03-13T00:50:12Z")
I'm trying to group multiline messages belonging to the same event. Unfortunately they don't appear to match the style of the examples in multiline-examples.html#\_examples\_of\_multiline\_configuration. Each event has a unique 9-character alphanumeric value that should be easily isolated, and I can't figure out how to do that. The number of lines per event can be dynamic so isolating "New ApplicationName SOAP request" as the start and line counting won't work. I've trouble a couple approaches, most recently: ```auto multiline.type: while_pattern multiline.pattern: '^.*[ApplicationName:(.*)].*$' multiline.negate: false multiline.match: after ``` Example of the logfile entry: ```auto [2024-03-10 00:07:16] [127.0.0.1:35822] [ApplicationName:D6NIHEW3] New ApplicationName SOAP request [2024-03-10 00:07:16] [127.0.0.1:35822] [ApplicationName:D6NIHEW3] > Username: some_user_name [2024-03-10 00:07:16] [127.0.0.1:35822] [ApplicationName:D6NIHEW3] > Password: xxxxxxxxxxx [2024-03-10 00:07:16] [127.0.0.1:35822] [ApplicationName:D6NIHEW3] > Client ID: some_ip_address [2024-03-10 00:07:16] [127.0.0.1:35822] [ApplicationName:D6NIHEW3] > Source IP: some_ip_address [2024-03-10 00:07:16] [127.0.0.1:35822] [ApplicationName:D6NIHEW3] Registered ApplicationName request [2024-03-10 00:07:16] [127.0.0.1:35822] [ApplicationName:D6NIHEW3] User invalid or not found [2024-03-10 00:07:16] [127.0.0.1:35822] [ApplicationName:D6NIHEW3] Sent failure response ``` Thank you! ---
**Author:** ![San9](https://avatars.discourse-cdn.com/v4/letter/s/a9adbd/32.png) [@San9](https://discuss.elastic.co/u/San9)\ **Post date:** [March 13, 2024, 9:46am UTC](https://discuss.elastic.co/t/filebeat-multiline-based-on-capture-group/355267/2 "2024-03-13T09:46:38Z")
Hi, what version of filebeat? what type are you using? I had a similar problem, perhaps the problem is in the config. I finally got multiline working. To understand all the pictures, it would be nice to see a little more of the log in order to understand the beginning and end of the multiline ---
**Author:** ![Operate6423](https://avatars.discourse-cdn.com/v4/letter/o/3ab097/32.png) [@Operate6423](https://discuss.elastic.co/u/Operate6423)\ **Post date:** [March 13, 2024, 4:39pm UTC](https://discuss.elastic.co/t/filebeat-multiline-based-on-capture-group/355267/3 "2024-03-13T16:39:29Z")
My settings/version are: ```auto # filebeat version filebeat version 8.6.2 (amd64), libbeat 8.6.2 [9b77c2c135c228c2eedc310f6e975bb1a76169b1 built 2023-02-12 04:37:19 +0000 UTC] filebeat.yml: filebeat.inputs: - type: filestream id: my_filebeat_id_for_this enabled: true paths: - /root/test_file.log multiline.type: while_pattern multiline.pattern: '^.*[ApplicationName:(.*)].*$' multiline.negate: false multiline.match: after output.console: pretty: true ``` ---
**Author:** ![San9](https://avatars.discourse-cdn.com/v4/letter/s/a9adbd/32.png) [@San9](https://discuss.elastic.co/u/San9)\ **Post date:** [March 14, 2024, 7:58am UTC](https://discuss.elastic.co/t/filebeat-multiline-based-on-capture-group/355267/4 "2024-03-14T07:58:22Z")
Hi. I'm using following config ```auto - type: filestream id: tomcat-catalina tags: ["tomcat"] enabled: true ignore_older: 24h paths: - /var/log/tomcat/catalina.*.log parsers: - multiline: type: pattern pattern: '^\w{3}\s[0-9]{2}' negate: true match: after ``` I think there is also a question with the log format itself. You need to understand the structure of the logs so that you can choose the right configuration and pattern ---
**Author:** ![Operate6423](https://avatars.discourse-cdn.com/v4/letter/o/3ab097/32.png) [@Operate6423](https://discuss.elastic.co/u/Operate6423)\ **Post date:** [April 1, 2024, 11:41pm UTC](https://discuss.elastic.co/t/filebeat-multiline-based-on-capture-group/355267/5 "2024-04-01T23:41:24Z")
I've truncated the log example and am still struggling to get the result I want. I get filebeat to match patterns, but it globs them all together into a single message. Using this as a test log format: ```auto [ApplicationName:D6NIHEW3] New ApplicationName SOAP request [ApplicationName:D6NIHEW3] > Username: some_user_name [ApplicationName:D6NIHEW3] > Password: xxxxxxxxxxx [ApplicationName:D6NIHEW3] > Client ID: some_ip_address [ApplicationName:D6NIHEW3] > Source IP: some_ip_address [ApplicationName:D6NIHEW3] Registered ApplicationName request [ApplicationName:D6NIHEW3] User invalid or not found [ApplicationName:D6NIHEW3] Sent failure response [ApplicationName:G6H4564S] New ApplicationName SOAP request [ApplicationName:G6H4564S] > Username: some_other_user_name [ApplicationName:G6H4564S] > Password: xxxxxxxxxxx [ApplicationName:G6H4564S] > Client ID: some_ip_address [ApplicationName:G6H4564S] > Source IP: some_ip_address [ApplicationName:G6H4564S] Registered ApplicationName request [ApplicationName:G6H4564S] User invalid or not found [ApplicationName:G6H4564S] Sent failure response ``` Guess I'm wondering if I can have each while\_pattern track the unique event ID's D6NIHEW3 & G6H4564S to log as different messages. Latest test config: ```auto filebeat.inputs: - type: filestream id: id_I_am_using enabled: true paths: - '/tmp/13' parsers: - multiline: type: while_pattern pattern: '^\[ApplicationName:[A-Z0-9]+\]' negate: false match: after ``` ---
**Author:** ![Operate6423](https://avatars.discourse-cdn.com/v4/letter/o/3ab097/32.png) [@Operate6423](https://discuss.elastic.co/u/Operate6423)\ **Post date:** [April 1, 2024, 11:49pm UTC](https://discuss.elastic.co/t/filebeat-multiline-based-on-capture-group/355267/6 "2024-04-01T23:49:15Z")
Actually, I may have something.. if I set the pattern to include "New openotpSimpleLogin SOAP request", and set negate: true, I get the result I want. ```auto pattern: '^\[ApplicationName:[A-Z0-9]+\] New openotpSimpleLogin SOAP request$' ``` However, I feel there should be a way to just match a unique [[:alnum:]] in every message and treat it as a multiline. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 30, 2024, 1:49am UTC](https://discuss.elastic.co/t/filebeat-multiline-based-on-capture-group/355267/7 "2024-04-30T01:49:21Z")
Closed on April 30, 2024, 1:49am UTC This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.