# Filebeat multiline problem with logs containing backticks

**URL:** <https://discuss.elastic.co/t/filebeat-multiline-problem-with-logs-containing-backticks/73357>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [January 31, 2017, 12:14pm UTC](https://discuss.elastic.co/t/filebeat-multiline-problem-with-logs-containing-backticks/73357 "2017-01-31T12:14:22Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Babadofar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/babadofar/32/44803_2.png) [@Babadofar](https://discuss.elastic.co/u/Babadofar)\
**Post date:** [January 31, 2017, 12:14pm UTC](https://discuss.elastic.co/t/filebeat-multiline-problem-with-logs-containing-backticks/73357/1 "2017-01-31T12:14:22Z")

</div>

I am trying to get multline logs into filebeat from an [ASP.Net](http://ASP.Net) MVC solution. I noticed a problem when testing out the multiline pattern due to the logs contain ``` `backtick` characters, but hoped it was just due to the multiline test debugger. It turns out there actually is a problem. Anyone knows what to do with logs that contain backticks?

The error messages I get are `Provided Grok expressions do not match field value`

Sample log with several backticks:

```
System.Web.HttpException (0x80004005): A public action method 'Post' was not found on controller 'BABADOFAR.Controllers.LoginPageController'.
  at System.Web.Mvc.Controller.HandleUnknownAction(String actionName)
 at EPiServer.Web.Mvc.ActionControllerBase.HandleUnknownAction(String actionName)
at System.Web.Mvc.Controller.<BeginExecuteCore>b__1d(IAsyncResult asyncResult, ExecuteCoreState innerState)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegat e(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.EndExecuteCore(IAsyncResult asyncResult)
 at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.EndExecute(IAsyncResult asyncResult)
at System.Web.Mvc.MvcHandler.<BeginProcessRequest>b__5(IAsyncResult asyncResult, ProcessRequestState innerState)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult)
at System.Web.Mvc.MvcHandler.EndProcessRequest(IAsyncResult asyncResult)
at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute ()
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)
```

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [January 31, 2017, 3:03pm UTC](https://discuss.elastic.co/t/filebeat-multiline-problem-with-logs-containing-backticks/73357/2 "2017-01-31T15:03:13Z")

</div>

Filebeat doesn't use grok so perhaps this is a Logstash error and this should be a post in the Logstash topic.

However Filebeat can do multiline on its own. See

- [https://www.elastic.co/guide/en/beats/filebeat/current/configuration-filebeat-options.html#multiline](https://www.elastic.co/guide/en/beats/filebeat/current/configuration-filebeat-options.html#multiline)
- [https://www.elastic.co/guide/en/beats/filebeat/current/multiline-examples.html](https://www.elastic.co/guide/en/beats/filebeat/current/multiline-examples.html)

---

<div class="post-metadata">

**Author:** ![Babadofar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/babadofar/32/44803_2.png) [@Babadofar](https://discuss.elastic.co/u/Babadofar)\
**Post date:** [February 1, 2017, 8:15am UTC](https://discuss.elastic.co/t/filebeat-multiline-problem-with-logs-containing-backticks/73357/3 "2017-02-01T08:15:02Z")

</div>

Thanks for your help but I'm sorry, I am actually using filebeat.  
This is the multiline pattern I'm using.

```
multiline.match: after
multiline.negate: true
multiline.pattern: "^[0-9]{4}-[0-9]{2}-[0-9]{2}"
```

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [February 1, 2017, 1:16pm UTC](https://discuss.elastic.co/t/filebeat-multiline-problem-with-logs-containing-backticks/73357/4 "2017-02-01T13:16:20Z")

</div>

this multiline config just looks for a date at the beginning of the line. Otherwise filebeat doesn't care about the content besides the final new-line characters. Content is send to logstash as is.

---

<div class="post-metadata">

**Author:** ![Babadofar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/babadofar/32/44803_2.png) [@Babadofar](https://discuss.elastic.co/u/Babadofar)\
**Post date:** [February 2, 2017, 4:52pm UTC](https://discuss.elastic.co/t/filebeat-multiline-problem-with-logs-containing-backticks/73357/5 "2017-02-02T16:52:13Z")

</div>

I'm using elasticsearch pipeline, though, not logstash. The lines are split where the backtick character occurs in the logs. I will register this as a bug then.

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [February 3, 2017, 3:31pm UTC](https://discuss.elastic.co/t/filebeat-multiline-problem-with-logs-containing-backticks/73357/6 "2017-02-03T15:31:21Z")

</div>

Do you have a more 'complete' log with timestamps and such? Maybe share more details of your filebeat and ingest node pipeline config? Have you checked if filebeat is splitting the event on the backtick (e.g. output to file or stdout)?

With the amount of information we've so far I can not tell if it's a bug in beats or ingest node.

---

<div class="post-metadata">

**Author:** ![Babadofar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/babadofar/32/44803_2.png) [@Babadofar](https://discuss.elastic.co/u/Babadofar)\
**Post date:** [February 3, 2017, 6:22pm UTC](https://discuss.elastic.co/t/filebeat-multiline-problem-with-logs-containing-backticks/73357/7 "2017-02-03T18:22:13Z")

</div>

I tried now without using the ingest pipeline, and I get the complete messages, new lines, weird characters and all. So there is most likely something wrong in my grok. Sorry about the fuzz!

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [February 4, 2017, 2:33pm UTC](https://discuss.elastic.co/t/filebeat-multiline-problem-with-logs-containing-backticks/73357/8 "2017-02-04T14:33:16Z")

</div>

Thanks for reporting back. Would still be interesting to learn/investigate if this might be a potential bug in ES or not. I'd recommend the ingest node simulate API with failing cases.

---

<div class="post-metadata">

**Author:** ![Babadofar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/babadofar/32/44803_2.png) [@Babadofar](https://discuss.elastic.co/u/Babadofar)\
**Post date:** [February 6, 2017, 12:00pm UTC](https://discuss.elastic.co/t/filebeat-multiline-problem-with-logs-containing-backticks/73357/9 "2017-02-06T12:00:42Z")

</div>

I finally found the problem. It was a misconfiguration in the filebeat.yml prospectors section. I had two log prospectors, but only one multiline section, somehow I was under the impression that the multiline settings would apply to all prospectors. Yet another PEBKAC 🙂

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [February 6, 2017, 3:51pm UTC](https://discuss.elastic.co/t/filebeat-multiline-problem-with-logs-containing-backticks/73357/10 "2017-02-06T15:51:05Z")

</div>

Phew... no bug. Thanks for reporting back.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 6, 2017, 3:51pm UTC](https://discuss.elastic.co/t/filebeat-multiline-problem-with-logs-containing-backticks/73357/11 "2017-03-06T15:51:09Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
