# Filebeat's Multiline not grouping properly **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [September 15, 2018, 3:56pm UTC](https://discuss.elastic.co/t/filebeats-multiline-not-grouping-properly/148711 "2018-09-15T15:56:26Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![kellen.hopwood](https://avatars.discourse-cdn.com/v4/letter/k/9de053/32.png) [@kellen.hopwood](https://discuss.elastic.co/u/kellen.hopwood)\ **Post date:** [September 15, 2018, 3:56pm UTC](https://discuss.elastic.co/t/filebeats-multiline-not-grouping-properly/148711/1 "2018-09-15T15:56:26Z")
Hi All, I am using filebeats to try and pull from a text file that contains http logs, the logs are generally not uniform, so I used powershell to create an end of file marker. However when I then try and send the logs into elastic and later kibana, I get one or two massive logs, not the individual components. ``` filebeat.inputs: - type: log enabled: true paths: - 'C:\Program Files\filebeat\logfile.txt' multiline: pattern: 'XXXXX' negate: true match: after output.logstash: hosts: ["localhost:5044"] ``` The data looks like this. ``` GET https://scontent-iad3-1.xx.fbcdn.net/v/t39.2093- 6/36882171_273786243171121_8594480363911249920_n.srt? _nc_cat=0&oh=6dae5ff3122dd34ae1cb4dfccfda7779&oe=5C307E15 HTTP/1.1 Host: scontent-iad3-1.xx.fbcdn.net Connection: keep-alive Origin: https://www.facebook.com User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.92 Safari/537.36 Accept: */* Referer: https://www.facebook.com/ Accept-Encoding: gzip, deflate, br Accept-Language: en-US,en;q=0.9 AAAAA XXXXX HTTP/1.1 200 OK Last-Modified: Tue, 10 Jul 2018 23:39:25 GMT Content-Type: text/srt Timing-Allow-Origin: * Access-Control-Allow-Origin: * Expires: Tue, 18 Sep 2018 23:54:05 GMT Cache-Control: max-age=1209600, no-transform Date: Sat, 15 Sep 2018 15:14:56 GMT Access-Control-Expose-Headers: X-FB-CEC-Video-Limit Connection: keep-alive Content-Length: 46280 AAAAA XXXXX ``` My original attempts to have it sort the data have all gone, rather abysmally, I was hoping someone here could give me some quick pointers to get more acclimated. Thanks! ---
**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\ **Post date:** [September 17, 2018, 11:54am UTC](https://discuss.elastic.co/t/filebeats-multiline-not-grouping-properly/148711/2 "2018-09-17T11:54:16Z")
Hi @kellen.hopwood and welcome 🙂 Your configuration looks correct for the example data you posted. What I see that could be wrong is the indentation of the configuration, `enabled`, `paths` and `multiline` must be at the same level as `type`, i.e. ```auto filebeat.inputs: - type: log enabled: true paths: - 'C:\Program Files\filebeat\logfile.txt' multiline: pattern: 'XXXXX' negate: true match: after ``` Could you review the indentation of your configuration file? ---
**Author:** ![kellen.hopwood](https://avatars.discourse-cdn.com/v4/letter/k/9de053/32.png) [@kellen.hopwood](https://discuss.elastic.co/u/kellen.hopwood)\ **Post date:** [September 17, 2018, 7:45pm UTC](https://discuss.elastic.co/t/filebeats-multiline-not-grouping-properly/148711/3 "2018-09-17T19:45:33Z")
Hi @jsoriano, I checked the indentation, and it was incorrect in my post, I corrected it, but it looks like the indentation is correct in my config file. ---
**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\ **Post date:** [September 18, 2018, 12:32pm UTC](https://discuss.elastic.co/t/filebeats-multiline-not-grouping-properly/148711/4 "2018-09-18T12:32:11Z")
@kellen.hopwood I saw you edited the sample data, take into account that with your configuration the pattern `XXXXX` must appear as the first line of your multiline event. Another option you could try to avoid needing to add these placeholders is to use a pattern that matches with the usual requests you have, i.e. something like this: ```auto multiline: pattern: '^(GET|HTTP/)' negate: true match: after ``` ---
**Author:** ![kellen.hopwood](https://avatars.discourse-cdn.com/v4/letter/k/9de053/32.png) [@kellen.hopwood](https://discuss.elastic.co/u/kellen.hopwood)\ **Post date:** [September 18, 2018, 2:18pm UTC](https://discuss.elastic.co/t/filebeats-multiline-not-grouping-properly/148711/5 "2018-09-18T14:18:13Z")
Hi @jsoriano, I have been digging through my logs, and even when attempting to match the first line, with HTTP or POST or HEAD, GET, ect, I still get the entire file being sent pretty much all in one big log, and I was hoping that I could separate each packet. However with Negate True I can't seem to get the individual packets what are being separated by the matching regex. With negate off, I get each line of the log sent individually. It seems like it maybe a regex parse issue? I think I'm going to try and insert an EOF at the end of every file and the use the flush command on that. ---
**Author:** ![kellen.hopwood](https://avatars.discourse-cdn.com/v4/letter/k/9de053/32.png) [@kellen.hopwood](https://discuss.elastic.co/u/kellen.hopwood)\ **Post date:** [September 18, 2018, 2:35pm UTC](https://discuss.elastic.co/t/filebeats-multiline-not-grouping-properly/148711/6 "2018-09-18T14:35:18Z")
I tried to insert a flush, point at the end of each log, however it still isn't recognizing either point, it's just grabbing all the data because it registers it all as a false, and because negate is on, it inverts that and registers it all as true. Any advice @jsoriano? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 16, 2018, 2:35pm UTC](https://discuss.elastic.co/t/filebeats-multiline-not-grouping-properly/148711/7 "2018-10-16T14:35:24Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.