# Multiline pattern filebeat

**URL:** <https://discuss.elastic.co/t/multiline-pattern-filebeat/70474>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [January 3, 2017, 7:14pm UTC](https://discuss.elastic.co/t/multiline-pattern-filebeat/70474 "2017-01-03T19:14:31Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![muralibala](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/muralibala/32/14224_2.png) [@muralibala](https://discuss.elastic.co/u/muralibala)\
**Post date:** [January 3, 2017, 7:14pm UTC](https://discuss.elastic.co/t/multiline-pattern-filebeat/70474/1 "2017-01-03T19:14:31Z")

</div>

I have the filebeat prospector setup for a custom log file. The errors are in multiple lines.

Ex:  
####\<Jan 1, 2017 2:31:53 PM EST\> \<[ACTIVE] ExecuteThread: '30' for queue: 'weblogic.kernel.Default (self-tuning)'\> \<\> \<\> \<\> \<1483299113805\> \<[ServletContext@371700326[app:TC module:DeltekTC path:null spec-version:null]] Servlet failed with an Exception  
java.lang.NumberFormatException: null  
at java.lang.Integer.parseInt(Integer.java:454)  
at java.lang.Integer.parseInt(Integer.java:527)  
at jsp\_servlet.\_com.\_deltek.\_tc.\_ts.\_\_timesheet2.\_jspService(\_\_timesheet2.java:1545)  
at weblogic.servlet.jsp.JspBase.service(JspBase.java:34)  
at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:242)  
at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:216)  
at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:132)  
at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:338)  
at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:221)  
at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.wrapRun(WebAppServletContext.java:3284)  
at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3254)  
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)  
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:120)  
at weblogic.servlet.provider.WlsSubjectHandle.run(WlsSubjectHandle.java:57)  
at weblogic.servlet.internal.WebAppServletContext.doSecuredExecute(WebAppServletContext.java:2163)  
at weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2089)  
at weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2074)  
at weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1513)  
at weblogic.servlet.provider.ContainerSupportProviderImpl$WlsRequestExecutor.run(ContainerSupportProviderImpl.java:254)  
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:256)  
at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)

> 

My filebeat.prospectors looks like this:

input\_type: log

# Paths that should be crawled and fetched. Glob based paths.

paths:  
- C:\logs\DTServer.log

document\_type: DTLog

ignore\_older: 5h  
multiline.pattern: '\[1\]'  
multiline.negate: false  
multiline.match: after

So multiline.pattern: '\[2\]' does not work. Multiple empty log entries are being created.

What would be my regex for the multiline pattern to get everything between ####\< and \>

TIA

* * *

1. [:space:] 

2. [:space:]

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [January 3, 2017, 9:05pm UTC](https://discuss.elastic.co/t/multiline-pattern-filebeat/70474/2 "2017-01-03T21:05:47Z")

</div>

can you please format you logs and config file using the `</>` button?

Can you add some more log lines for us to get a better idea about the log files structure? E.g. if a log-entry always starts with `<`, why not use this one for filtering?

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [January 5, 2017, 12:15pm UTC](https://discuss.elastic.co/t/multiline-pattern-filebeat/70474/3 "2017-01-05T12:15:30Z")

</div>

The playground here is very useful to test your regexp expressions: [https://www.elastic.co/guide/en/beats/filebeat/5.1/multiline-examples.html#\_testing\_your\_regexp\_pattern\_for\_multiline](https://www.elastic.co/guide/en/beats/filebeat/5.1/multiline-examples.html#_testing_your_regexp_pattern_for_multiline)

---

<div class="post-metadata">

**Author:** ![muralibala](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/muralibala/32/14224_2.png) [@muralibala](https://discuss.elastic.co/u/muralibala)\
**Post date:** [January 5, 2017, 2:44pm UTC](https://discuss.elastic.co/t/multiline-pattern-filebeat/70474/5 "2017-01-05T14:44:34Z")

</div>

All log enteries are in the following format:

```
####<Jan 5, 2017 8:26:51 AM EST> <Error> <com.deltek.tc.framework> <Server> <TEServer> <[ACTIVE] ExecuteThread: '6' for queue: 'weblogic.kernel.Default (self-tuning)'> <1382934[0$X]WESTAT> <> <> <1483622811881> <BEA-000000> <FrontServlet:doPost:
********************Exception 0********************
java.lang.NullPointerException
	at com.deltek.tc.framework.EssAppController.doPost(EssAppController.java:44)
	at com.deltek.tc.framework.Controller.doGet(Controller.java:32)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:844)
	at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:242)
	at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:216)
	at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:132)
	at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:338)
	at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:221)
	at weblogic.servlet.internal.RequestDispatcherImpl.invokeServlet(RequestDispatcherImpl.java:567)
	at weblogic.servlet.internal.RequestDispatcherImpl.forward(RequestDispatcherImpl.java:263)
	at com.deltek.tc.framework.RequestProcessor.doPost(RequestProcessor.java:42)
	at com.deltek.tc.framework.RequestProcessor.doGet(RequestProcessor.java:28)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:844)
	at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:242)
	at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:216)
	at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:132)
	at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:338)
	at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:221)
	at weblogic.servlet.internal.RequestDispatcherImpl.invokeServlet(RequestDispatcherImpl.java:567)
	at weblogic.servlet.internal.RequestDispatcherImpl.forward(RequestDispatcherImpl.java:263)
	at com.deltek.tc.framework.FrontServlet.doPost(FrontServlet.java:292)
	at com.deltek.tc.framework.FrontServlet.doGet(FrontServlet.java:316)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:844)
	at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:242)
	at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:216)
	at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:132)
	at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:338)
	at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:221)
	at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.wrapRun(WebAppServletContext.java:3284)
	at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3254)
	at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
	at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:120)
	at weblogic.servlet.provider.WlsSubjectHandle.run(WlsSubjectHandle.java:57)
	at weblogic.servlet.internal.WebAppServletContext.doSecuredExecute(WebAppServletContext.java:2163)
	at weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2089)
	at weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2074)
	at weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1513)
	at weblogic.servlet.provider.ContainerSupportProviderImpl$WlsRequestExecutor.run(ContainerSupportProviderImpl.java:254)
	at weblogic.work.ExecuteThread.execute(ExecuteThread.java:256)
	at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)

>
```

---

<div class="post-metadata">

**Author:** ![muralibala](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/muralibala/32/14224_2.png) [@muralibala](https://discuss.elastic.co/u/muralibala)\
**Post date:** [January 5, 2017, 2:45pm UTC](https://discuss.elastic.co/t/multiline-pattern-filebeat/70474/6 "2017-01-05T14:45:37Z")

</div>

So the log enteries always starts with `####<` and ends with `>`

---

<div class="post-metadata">

**Author:** ![muralibala](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/muralibala/32/14224_2.png) [@muralibala](https://discuss.elastic.co/u/muralibala)\
**Post date:** [January 5, 2017, 2:50pm UTC](https://discuss.elastic.co/t/multiline-pattern-filebeat/70474/7 "2017-01-05T14:50:12Z")

</div>

Awesome. Will give that a shot.

---

<div class="post-metadata">

**Author:** ![muralibala](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/muralibala/32/14224_2.png) [@muralibala](https://discuss.elastic.co/u/muralibala)\
**Post date:** [January 5, 2017, 6:42pm UTC](https://discuss.elastic.co/t/multiline-pattern-filebeat/70474/8 "2017-01-05T18:42:01Z")

</div>

Steffens,

This worked:

```
  multiline.pattern: '<*'
  multiline.negate: false
  multiline.match: after
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 2, 2017, 6:42pm UTC](https://discuss.elastic.co/t/multiline-pattern-filebeat/70474/9 "2017-02-02T18:42:28Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.