# Problem with multiline filebeat docker prospector (,\\n)

**URL:** <https://discuss.elastic.co/t/problem-with-multiline-filebeat-docker-prospector-n/122261>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [March 2, 2018, 1:43pm UTC](https://discuss.elastic.co/t/problem-with-multiline-filebeat-docker-prospector-n/122261 "2018-03-02T13:43:58Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![meril](https://avatars.discourse-cdn.com/v4/letter/m/dec6dc/32.png) [@meril](https://discuss.elastic.co/u/meril)\
**Post date:** [March 2, 2018, 1:43pm UTC](https://discuss.elastic.co/t/problem-with-multiline-filebeat-docker-prospector-n/122261/1 "2018-03-02T13:43:58Z")

</div>

Hi  
I configured filebeat 6.2.1 like this:

```
filebeat.prospectors:
- type: docker
  document_type: eoclog4j
  paths:
  - '/var/lib/docker/containers/*/*.log'
  containers.ids:
  - '*'
  multiline:
    pattern: '([0-9]{4}-[0-9]{2}-[0-9]{2})|(\[[0-9]{2}.[0-9]{2}.[0-9]{4})'
    negate: true
    match: after
  fields_under_root: true
  fields:
    source_system: filebeat_docker
  processors:
  - add_docker_metadata: ~

```

Everything seems ok except that sometime multiline fails. It seems to me (I know this is strange but I cannot find other explanations) that this happens when lines ends with: ,\n  
Example:

This went wrong (lines are divided):  
{"log":"2018-03-01T14:34:21.032-0000 - ERROR - [server] - Error in blablblabla:  
'2018-02-23 13:43:59.653165000 +0000',\n","stream":"stdout","time":"2018-03-01T14:34:21.145493486Z"}  
{"log":" 'blablabla': '2018-02-23 13:43:59.465964600 +0000',\n","stream":"stdout","time":"2018-03-01T14:34:21  
.14552074Z"}

This works OK (lines goes in same event):  
{"log":" at blablabla\n","stream":"stdout","time":"2018-03-01T14:34:21.145679703Z"}  
{"log":" at blablabla (events.js:126:13)\n","stream":"stdout","time":"2018-03-01T14:34:21.145683059Z"}  
{"log":" at blablabla (events.js:214:7)\n","stream":"stdout","time":"2018-03-01T14:34:21.145686123Z"}

Any ideas?  
Many thanks

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [March 3, 2018, 11:39pm UTC](https://discuss.elastic.co/t/problem-with-multiline-filebeat-docker-prospector-n/122261/2 "2018-03-03T23:39:56Z")

</div>

To figure out what goes wrong with the regexp I often use the go playground linked here to test the regexp: [https://www.elastic.co/guide/en/beats/filebeat/6.2/\_test\_your\_regexp\_pattern\_for\_multiline.html](https://www.elastic.co/guide/en/beats/filebeat/6.2/_test_your_regexp_pattern_for_multiline.html)

---

<div class="post-metadata">

**Author:** ![meril](https://avatars.discourse-cdn.com/v4/letter/m/dec6dc/32.png) [@meril](https://discuss.elastic.co/u/meril)\
**Post date:** [March 4, 2018, 10:49am UTC](https://discuss.elastic.co/t/problem-with-multiline-filebeat-docker-prospector-n/122261/3 "2018-03-04T10:49:36Z")

</div>

Thank you ruflin.👏👏  
I did what you suggested and I found the problem.  
I missed the ^ character in beginning of my pattern so it matched also wrong lines.  
Many thanks

---

<div class="post-metadata">

**Author:** ![meril](https://avatars.discourse-cdn.com/v4/letter/m/dec6dc/32.png) [@meril](https://discuss.elastic.co/u/meril)\
**Post date:** [March 4, 2018, 10:57am UTC](https://discuss.elastic.co/t/problem-with-multiline-filebeat-docker-prospector-n/122261/4 "2018-03-04T10:57:33Z")

</div>

By the way... what about to put go playground tool you suggested in kibana dev tools (together grok tester) ? It could be useful.  
Thanks again

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [March 12, 2018, 6:42am UTC](https://discuss.elastic.co/t/problem-with-multiline-filebeat-docker-prospector-n/122261/5 "2018-03-12T06:42:01Z")

</div>

I quite like the idea. The main issue here is that regexp != regexp. So we would need to have to go regexp executed in JS somehow. I will definitively keep the idea in mind.

We are also in the progress to make testing easier directly in Filebeat.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 9, 2018, 6:42am UTC](https://discuss.elastic.co/t/problem-with-multiline-filebeat-docker-prospector-n/122261/6 "2018-04-09T06:42:03Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.