# Handling multiline log with a timestamp on each line **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [January 30, 2020, 5:00pm UTC](https://discuss.elastic.co/t/handling-multiline-log-with-a-timestamp-on-each-line/217248 "2020-01-30T17:00:43Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![ricky.kwan.ix](https://avatars.discourse-cdn.com/v4/letter/r/a183cd/32.png) [@ricky.kwan.ix](https://discuss.elastic.co/u/ricky.kwan.ix)\ **Post date:** [January 30, 2020, 5:00pm UTC](https://discuss.elastic.co/t/handling-multiline-log-with-a-timestamp-on-each-line/217248/1 "2020-01-30T17:00:43Z")
Hi, I'm trying to combine multiple log lines into a single line, but the issue is that each line has a timestamp (among other things) on it. ```nohighlight Jan 30 2020 16:52:16 GMT: INFO (info): (hist.c:240) histogram dump: {test}-write (2366144456 total) msec Jan 30 2020 16:52:16 GMT: INFO (info): (hist.c:257) (00: 2348746349) (01: 0002309726) (02: 0002863034) (03: 0011897069) Jan 30 2020 16:52:16 GMT: INFO (info): (hist.c:257) (04: 0000212507) (05: 0000095451) (06: 0000017783) (07: 0000002284) Jan 30 2020 16:52:16 GMT: INFO (info): (hist.c:257) (08: 0000000236) (09: 0000000011) (10: 0000000004) (11: 0000000001) Jan 30 2020 16:52:16 GMT: INFO (info): (hist.c:266) (12: 0000000001) ``` As such, the combined line ends up having the timestamp (and the other stuff) multiple times. **Has anyone had a similar situation before and worked around it?** I was hoping that `multiline.pattern` could have capturing groups or something to keep the relevant parts but that doesn't seem to be the case. I'm running 6.4 (no new multiline-related configs in 7.X) with the following settings: ```nohighlight - fields_under_root: true paths: - /var/log/aerospike/aerospike.log type: log multiline.pattern: "[a-zA-Z]{3} \\d{1,2} \\d{4} \\d{2}:\\d{2}:\\d{2}(\\.\\d+)? [A-Z]{3}: [A-Z]+ \\([a-z_-]+\\): \\(hist\\.c:\\d+\\)(\\s+\\(\\d+: \\d+\\))+" multiline.negate: false multiline.match: after ``` ---
**Author:** ![Kaiyan\_Sheng](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kaiyan_sheng/32/38247_2.png) [@Kaiyan\_Sheng](https://discuss.elastic.co/u/Kaiyan_Sheng)\ **Post date:** [January 30, 2020, 10:07pm UTC](https://discuss.elastic.co/t/handling-multiline-log-with-a-timestamp-on-each-line/217248/2 "2020-01-30T22:07:00Z")
Hi @ricky.kwan.ix, this is a very interesting problem to solve. Maybe [https://www.elastic.co/guide/en/beats/filebeat/current/\_examples\_of\_multiline\_configuration.html#\_application\_events](https://www.elastic.co/guide/en/beats/filebeat/current/_examples_of_multiline_configuration.html#_application_events) will help?! The `start new event` in your case would be `histogram dump:` and `end event` would be `(12: ...)`? ---
**Author:** ![ricky.kwan.ix](https://avatars.discourse-cdn.com/v4/letter/r/a183cd/32.png) [@ricky.kwan.ix](https://discuss.elastic.co/u/ricky.kwan.ix)\ **Post date:** [January 30, 2020, 10:47pm UTC](https://discuss.elastic.co/t/handling-multiline-log-with-a-timestamp-on-each-line/217248/3 "2020-01-30T22:47:48Z")
Hi, I tried your suggestion with the same result. And maybe it wasn't clear in my original post what I wanted, so I edited it. Here is a truncated test result after applying the flush\_pattern config. ```nohighlight "message": "Jan 30 2020 22:43:04.934 GMT: INFO (info): (hist.c:240) histogram dump: {test}-write (11 total) msec\nJan 30 2020 22:43:04.934 GMT: INFO (info): (hist.c:266) (00: 0000000011)" ``` The middle of the line has the timestamp, loglevel, etc that I want to remove when combined. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 27, 2020, 10:47pm UTC](https://discuss.elastic.co/t/handling-multiline-log-with-a-timestamp-on-each-line/217248/4 "2020-02-27T22:47:54Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.