# Logstash multiline codec do not read all lines

**URL:** <https://discuss.elastic.co/t/logstash-multiline-codec-do-not-read-all-lines/313542>\
**Category:** Logstash\
**Created:** [September 2, 2022, 10:30am UTC](https://discuss.elastic.co/t/logstash-multiline-codec-do-not-read-all-lines/313542 "2022-09-02T10:30:15Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![alex\_96](https://avatars.discourse-cdn.com/v4/letter/a/ec9cab/32.png) [@alex\_96](https://discuss.elastic.co/u/alex_96)\
**Post date:** [September 2, 2022, 10:30am UTC](https://discuss.elastic.co/t/logstash-multiline-codec-do-not-read-all-lines/313542/1 "2022-09-02T10:30:15Z")

</div>

Hi I'm using multiline codec to concatenate lines into one event, here's an example of it:

```auto
# Time: 2022-08-29T07:43:30.314166Z
# User@Host: root[root] @ 1.1.1.1. [] Id: 111111
# Query_time: 0.019870 Lock_time: 0.000000 Rows_sent: 0 Rows_examined: 0
SET timestamp=31554231;
set global slow_query_log_file="/logs/mysql-log";
# Time: 2022-08-29T12:34:30.419218Z
# User@Host: app[usr] @ [2.2.2.2] Id: 2222222
# Query_time: 0.000037 Lock_time: 0.000000 Rows_sent: 0 Rows_examined: 0
use aaaaa;
SET timestamp=4141454;
SET autocommit=4;

```

this is my config:

```auto
input {
  file {
   path => "/home/user1/test.log"
   start_position => "beginning"
   sincedb_path => "/dev/null"
   codec => multiline {
   pattern => "^#%{SPACE}Time:%{SPACE}%{TIMESTAMP_ISO8601:time}"
   negate => true
   what => "previous"
  }
 }
}

```

Everything works, but the problem is that it does not read the second part of the log, namely everything that starts with

```auto
# Time: 2022-08-29T12:34:30.419218Z

```

however, if after "SET autocommit=4;" I add a similar timestamp "# Time: 2022-08-29....", then it will read everything up to "# Time: 2022-08-29....".  
The question is how to make it read everything to the end?

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [September 2, 2022, 12:55pm UTC](https://discuss.elastic.co/t/logstash-multiline-codec-do-not-read-all-lines/313542/2 "2022-09-02T12:55:21Z")

</div>

You need to set the `auto_flush_interval` to get the last event.

From the [documentation](https://www.elastic.co/guide/en/logstash/current/plugins-codecs-multiline.html#plugins-codecs-multiline-auto_flush_interval).

> The accumulation of multiple lines will be converted to an event when either a matching new line is seen or there has been no new data appended for this many seconds. No default. If unset, no auto\_flush.

Since you do not have anymore events, the last lines will not be converted to an event unless you set a timeout for the auto flush.

Try to add `auto_flush_interval => 60` in your configuration, this will convert the accumulated lines into an event if you do not have any more matching lines for 60 seconds.

---

<div class="post-metadata">

**Author:** ![alex\_96](https://avatars.discourse-cdn.com/v4/letter/a/ec9cab/32.png) [@alex\_96](https://discuss.elastic.co/u/alex_96)\
**Post date:** [September 2, 2022, 1:33pm UTC](https://discuss.elastic.co/t/logstash-multiline-codec-do-not-read-all-lines/313542/3 "2022-09-02T13:33:33Z")

</div>

Thank you very much, I should read the documentation more carefully)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 30, 2022, 1:33pm UTC](https://discuss.elastic.co/t/logstash-multiline-codec-do-not-read-all-lines/313542/4 "2022-09-30T13:33:38Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.