# Beats with multiline

**URL:** <https://discuss.elastic.co/t/beats-with-multiline/35957>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [November 30, 2015, 9:51pm UTC](https://discuss.elastic.co/t/beats-with-multiline/35957 "2015-11-30T21:51:20Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jeferson\_Martins](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jeferson_martins/32/5849_2.png) [@Jeferson\_Martins](https://discuss.elastic.co/u/Jeferson_Martins)\
**Post date:** [November 30, 2015, 9:51pm UTC](https://discuss.elastic.co/t/beats-with-multiline/35957/1 "2015-11-30T21:51:20Z")

</div>

can i use FileBeat With multiline?

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [November 30, 2015, 10:02pm UTC](https://discuss.elastic.co/t/beats-with-multiline/35957/2 "2015-11-30T22:02:14Z")

</div>

Not yet, but we are working on an implementation. Best is to follow this issue: [https://github.com/elastic/filebeat/issues/301](https://github.com/elastic/filebeat/issues/301)

---

<div class="post-metadata">

**Author:** ![abinay](https://avatars.discourse-cdn.com/v4/letter/a/838e76/32.png) [@abinay](https://discuss.elastic.co/u/abinay)\
**Post date:** [January 28, 2016, 10:57am UTC](https://discuss.elastic.co/t/beats-with-multiline/35957/3 "2016-01-28T10:57:48Z")

</div>

@ruflin by what time it will get available ??? We need this as we need to take care of the laravel logs stack traces.

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [January 28, 2016, 5:38pm UTC](https://discuss.elastic.co/t/beats-with-multiline/35957/4 "2016-01-28T17:38:02Z")

</div>

We currently plan to release 1.1 with this feature at the beginning of February.

---

<div class="post-metadata">

**Author:** ![Jeferson\_Martins](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jeferson_martins/32/5849_2.png) [@Jeferson\_Martins](https://discuss.elastic.co/u/Jeferson_Martins)\
**Post date:** [February 18, 2016, 2:39pm UTC](https://discuss.elastic.co/t/beats-with-multiline/35957/5 "2016-02-18T14:39:43Z")

</div>

Thanks

---

<div class="post-metadata">

**Author:** ![seefood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/seefood/32/7939_2.png) [@seefood](https://discuss.elastic.co/u/seefood)\
**Post date:** [February 21, 2016, 12:17pm UTC](https://discuss.elastic.co/t/beats-with-multiline/35957/6 "2016-02-21T12:17:41Z")

</div>

Works, in that I can configure it without getting complaints about bad syntax, but I can't get the output I expect, also seems lots of lines are lost. Maybe it's still in alpha 🙂

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [February 21, 2016, 5:15pm UTC](https://discuss.elastic.co/t/beats-with-multiline/35957/7 "2016-02-21T17:15:36Z")

</div>

@seefood Multiline was release with 1.1. Can you give some more details on your issue?

---

<div class="post-metadata">

**Author:** ![mmuszkow](https://avatars.discourse-cdn.com/v4/letter/m/f19dbf/32.png) [@mmuszkow](https://discuss.elastic.co/u/mmuszkow)\
**Post date:** [February 22, 2016, 1:37pm UTC](https://discuss.elastic.co/t/beats-with-multiline/35957/8 "2016-02-22T13:37:43Z")

</div>

Are grok patterns expanded?

I transformed my config from multiline codec that worked following [this](https://github.com/elastic/beats/issues/461) and [this](https://github.com/spujadas/elk-docker/issues/23) from:

```auto
codec => multiline {
      pattern => "^20%{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{HOUR}:%{MINUTE}:%{SECOND}"
      negate => true
      what => "previous"
    }

```

into:

```auto
multiline:
        pattern: "^20%{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{HOUR}:%{MINUTE}:%{SECOND}"
        negate: true
        match: after

```

and it's not splitting the log lines correctly. I see lines merged together when they shouldn't be. I also tried expanding the pattern into `[0-9]{4}-[0-9]{2}-[0-9]{2}...`, but it doesn't work neither.

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [February 22, 2016, 4:40pm UTC](https://discuss.elastic.co/t/beats-with-multiline/35957/9 "2016-02-22T16:40:54Z")

</div>

Filebeat does not support grok patterns. For "valid" regexp see [https://github.com/elastic/beats/issues/740](https://github.com/elastic/beats/issues/740)

---

<div class="post-metadata">

**Author:** ![mmuszkow](https://avatars.discourse-cdn.com/v4/letter/m/f19dbf/32.png) [@mmuszkow](https://discuss.elastic.co/u/mmuszkow)\
**Post date:** [February 23, 2016, 9:51am UTC](https://discuss.elastic.co/t/beats-with-multiline/35957/10 "2016-02-23T09:51:18Z")

</div>

So I ended with

```auto
multiline:
        pattern: "^[[:digit:]]{4}-[[:digit:]]{2}-[[:digit:]]{2}"
        negate: true
        match: after

```

and it seems to work OK. Thanks,

---

<div class="post-metadata">

**Author:** ![matthewcanty](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/matthewcanty/32/8291_2.png) [@matthewcanty](https://discuss.elastic.co/u/matthewcanty)\
**Post date:** [March 4, 2016, 4:12pm UTC](https://discuss.elastic.co/t/beats-with-multiline/35957/11 "2016-03-04T16:12:48Z")

</div>

This should be written down in the documentation somewhere. I've been trying for **hours** to figure out what this achieves.

Thanks so much for posting your solution.

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [March 5, 2016, 1:18am UTC](https://discuss.elastic.co/t/beats-with-multiline/35957/12 "2016-03-05T01:18:26Z")

</div>

@matthewcanty we're actually thinking about collecting and documenting configurations for different scenarios. Any multiline scenarios you are interested in (samples?) ?

---

<div class="post-metadata">

**Author:** ![matthewcanty](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/matthewcanty/32/8291_2.png) [@matthewcanty](https://discuss.elastic.co/u/matthewcanty)\
**Post date:** [March 7, 2016, 9:42am UTC](https://discuss.elastic.co/t/beats-with-multiline/35957/13 "2016-03-07T09:42:37Z")

</div>

This is the only one we've needed so far. The .Net stack trace:

```
2016-03-04 14:50:07,614 [16] ERROR MyService.Domain.WebApi.Controllers.GreetingsController - Error on saving greeting.
System.InvalidOperationException: Database does not exist
   at MyService.Persistence.NonMutatingDatabaseInitialiser`1.InitializeDatabase(TContext context)
   at System.Data.Entity.Internal.InternalContext.PerformInitializationAction(Action action)
   at System.Data.Entity.Internal.InternalContext.PerformDatabaseInitialization()
   at System.Data.Entity.Internal.RetryAction`1.PerformAction(TInput input)
   at System.Data.Entity.Internal.LazyInternalContext.InitializeDatabaseAction(Action`1 action)
   at System.Data.Entity.Internal.InternalContext.GetEntitySetAndBaseTypeForType(Type entityType)
   at System.Data.Entity.Internal.Linq.InternalSet`1.Initialize()
   at System.Data.Entity.Internal.Linq.InternalSet`1.get_InternalContext()
   at System.Data.Entity.Infrastructure.DbQuery`1.System.Linq.IQueryable.get_Provider()
   at System.Linq.Queryable.Where[TSource](IQueryable`1 source, Expression`1 predicate)
   at MyService.Persistence.BaseRepository`2.Find(Expression`1 predicate)
   at MyService.Domain.ApplicationsServices.GreetingsService.<SaveGreetingsAsync>d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at MyService.Domain.WebApi.Controllers.GreetingsController.<PostExternalAccounts>d__1.MoveNext()
2016-03-04 14:50:07,614 [16] ERROR MyService..........
```

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [March 8, 2016, 8:40am UTC](https://discuss.elastic.co/t/beats-with-multiline/35957/14 "2016-03-08T08:40:39Z")

</div>

@matthewcanty Thanks for providing your example. We will take it into account when updating our docs.  
@dedemorton FYI

---

<div class="post-metadata">

**Author:** ![joedissmeyer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/joedissmeyer/32/26163_2.png) [@joedissmeyer](https://discuss.elastic.co/u/joedissmeyer)\
**Post date:** [February 9, 2017, 2:10pm UTC](https://discuss.elastic.co/t/beats-with-multiline/35957/15 "2017-02-09T14:10:16Z")

</div>

Thank you for posting this solution! This resolved my issue as well.

One of our applications writes logs multiline where all starting log entries start with the date itself. For example:

2017-02-09 04:00:21.517 C010 .... blah blah blah  
2017-02-09 07:38:44.507 A5B4 .... we love all datas.... blah blah

The pattern you provided here works PERFECTLY. Thank you again @mmuszkow!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 9:50pm UTC](https://discuss.elastic.co/t/beats-with-multiline/35957/16 "2017-07-05T21:50:00Z")

</div>


