# Multiline cuts off character

**URL:** <https://discuss.elastic.co/t/multiline-cuts-off-character/132468>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [May 18, 2018, 10:46am UTC](https://discuss.elastic.co/t/multiline-cuts-off-character/132468 "2018-05-18T10:46:48Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![manuelk](https://avatars.discourse-cdn.com/v4/letter/m/dbc845/32.png) [@manuelk](https://discuss.elastic.co/u/manuelk)\
**Post date:** [May 18, 2018, 10:46am UTC](https://discuss.elastic.co/t/multiline-cuts-off-character/132468/1 "2018-05-18T10:46:48Z")

</div>

Hello,

we´ve a case where Multiline seems to cut off the first character of a string within Filebeat 6.2.4.

Mulitline setting:

> multiline.pattern: '^(\*.JOB)|Cooldown'  
> multiline.negate: true  
> multiline.match: after

Message in File:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/a/0/a072970d32b90413a7cfbaedfa71a3ee81332b27.png)

Filebeat output:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/f/e/fe4bda42dd65479a4d574a4ad7d81e7f78466e84.png)

But this only happens when the file gets updated. When the file gets initially loaded, this does not happen.  
Anyone a suggestions what could be the issue?

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [May 18, 2018, 8:47pm UTC](https://discuss.elastic.co/t/multiline-cuts-off-character/132468/2 "2018-05-18T20:47:23Z")

</div>

Is the cut off first character included included with the previous event?

Can you try running with multiline debug enabled. If there's any flush triggered by a timeout there will be [log message](https://github.com/elastic/beats/blob/v6.2.4/filebeat/harvester/reader/multiline.go#L126).

```auto
logging.level: debug
logging.selectors: [multiline]

```

---

<div class="post-metadata">

**Author:** ![manuelk](https://avatars.discourse-cdn.com/v4/letter/m/dbc845/32.png) [@manuelk](https://discuss.elastic.co/u/manuelk)\
**Post date:** [May 22, 2018, 8:53am UTC](https://discuss.elastic.co/t/multiline-cuts-off-character/132468/3 "2018-05-22T08:53:27Z")

</div>

Hey,

thanks for your reply. I created a log and it looks like there is an timeout:  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/f/5/f59323dab306962a1893fcaed2544931a7f1913f.png)

Could this cause the issue? Could also explain why it works on old files where all datapoints are already available..

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [May 22, 2018, 1:24pm UTC](https://discuss.elastic.co/t/multiline-cuts-off-character/132468/4 "2018-05-22T13:24:27Z")

</div>

With pre-existing log data contained in the files the issue won't occur because Filebeat is able to read multiline data as fast as it can. Filebeat does not have to wait for the start of the next multiline block which is what normally triggers it to consider the previous multiline block to be complete and send it in an event.

This is probably caused by the process writing the file to not be flushing frequently enough. To account for this you can increase the multiline timeout value in Filebeat (see [multiline.timeout](https://www.elastic.co/guide/en/beats/filebeat/current/multiline-examples.html#multiline)).

---

<div class="post-metadata">

**Author:** ![manuelk](https://avatars.discourse-cdn.com/v4/letter/m/dbc845/32.png) [@manuelk](https://discuss.elastic.co/u/manuelk)\
**Post date:** [May 22, 2018, 2:08pm UTC](https://discuss.elastic.co/t/multiline-cuts-off-character/132468/5 "2018-05-22T14:08:28Z")

</div>

Hey,

yes this process is a real "slow mover" so we´re getting events every 5 minutes. I tried to increase timeout up to 7 minutes but still facing the same problem.

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [May 23, 2018, 1:23pm UTC](https://discuss.elastic.co/t/multiline-cuts-off-character/132468/6 "2018-05-23T13:23:41Z")

</div>

Can you try a really long value just to see if it fixes the issue? Worst case -- the most recent event will be delayed a bit.

```auto
multiline.pattern: '^(*.JOB)|Cooldown'
multiline.negate: true
multiline.match: after
multline.timeout: 1h

```

Do your events have a predictable string that can be used to flush? If so you could try the `multiline.flush_pattern`. (original [pull request](https://github.com/elastic/beats/pull/4019#issuecomment-296114885) for the feature)

---

<div class="post-metadata">

**Author:** ![manuelk](https://avatars.discourse-cdn.com/v4/letter/m/dbc845/32.png) [@manuelk](https://discuss.elastic.co/u/manuelk)\
**Post date:** [May 24, 2018, 7:09am UTC](https://discuss.elastic.co/t/multiline-cuts-off-character/132468/7 "2018-05-24T07:09:25Z")

</div>

Unfortunately the extended timeout did not work either. There´s no predictable string for the flush\_pattern - message is quite dynamic except the start pattern.

Do you have somethingin mind why it´s just affecting the first character (and only if a new event is added) ?

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [May 25, 2018, 11:13pm UTC](https://discuss.elastic.co/t/multiline-cuts-off-character/132468/8 "2018-05-25T23:13:34Z")

</div>

> [@manuelk](#):
>
> Unfortunately the extended timeout did not work either.

Did the logs still say that the multiline timeout was reached? That's the only reason I can think of that would cause a split.

---

<div class="post-metadata">

**Author:** ![manuelk](https://avatars.discourse-cdn.com/v4/letter/m/dbc845/32.png) [@manuelk](https://discuss.elastic.co/u/manuelk)\
**Post date:** [May 30, 2018, 2:06pm UTC](https://discuss.elastic.co/t/multiline-cuts-off-character/132468/9 "2018-05-30T14:06:38Z")

</div>

Hey,

sorry for the late reply - Now the timeout disappeared in the error log. I can´t find any further error message but the issue with the missing char still occurs.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/4/0/406f0a69a5e4464ff81fa4349ade4bbbd20f8176.png)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 27, 2018, 4:06pm UTC](https://discuss.elastic.co/t/multiline-cuts-off-character/132468/10 "2018-06-27T16:06:41Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
