# Logstash 6.2.4 read\_to\_eof: no delimiter found in current chunk **URL:** \ **Category:** Logstash\ **Created:** [May 4, 2018, 11:09am UTC](https://discuss.elastic.co/t/logstash-6-2-4-read-to-eof-no-delimiter-found-in-current-chunk/130613 "2018-05-04T11:09:28Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![ld\_pvl](https://avatars.discourse-cdn.com/v4/letter/l/cdc98d/32.png) [@ld\_pvl](https://discuss.elastic.co/u/ld_pvl)\ **Post date:** [May 4, 2018, 11:09am UTC](https://discuss.elastic.co/t/logstash-6-2-4-read-to-eof-no-delimiter-found-in-current-chunk/130613/1 "2018-05-04T11:09:28Z")
Hi guys, I am using the new file input plugin 4.1.1 and keep getting the below warn message churning out numerous times - what is it about? ``` [WARN][filewatch.tailmode.handlers.grow] read_to_eof: no delimiter found in current chunk ``` Cheers, ---
**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\ **Post date:** [May 4, 2018, 11:14am UTC](https://discuss.elastic.co/t/logstash-6-2-4-read-to-eof-no-delimiter-found-in-current-chunk/130613/2 "2018-05-04T11:14:39Z")
That error has just been fixed today. update to v4.1.2. Apologies. ---
**Author:** ![ld\_pvl](https://avatars.discourse-cdn.com/v4/letter/l/cdc98d/32.png) [@ld\_pvl](https://discuss.elastic.co/u/ld_pvl)\ **Post date:** [May 4, 2018, 11:19am UTC](https://discuss.elastic.co/t/logstash-6-2-4-read-to-eof-no-delimiter-found-in-current-chunk/130613/3 "2018-05-04T11:19:15Z")
Thanks for the quick reply. One question unrelated to the topic (maybe I should create another thread) but is there any problem in 4.1.1 that causes the same log events to be sent causing multiple duplicate docs in elasticsearch? Because I am facing this issue after upgrade where I see numerous duplicate docs. ---
**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\ **Post date:** [May 4, 2018, 11:26am UTC](https://discuss.elastic.co/t/logstash-6-2-4-read-to-eof-no-delimiter-found-in-current-chunk/130613/4 "2018-05-04T11:26:15Z")
The error caused the same piece of content to continually be reprocessed - I assume that is where the duplicates come from. Are you using `/dev/null` for your sincedb path? I suggest you start from scratch rereading the files to a new index and then check for duplicates. If you see duplicates after this please open a new topic "File input v4.1.2 tailing - duplicate docs ingested" ---
**Author:** ![ld\_pvl](https://avatars.discourse-cdn.com/v4/letter/l/cdc98d/32.png) [@ld\_pvl](https://discuss.elastic.co/u/ld_pvl)\ **Post date:** [May 4, 2018, 11:26am UTC](https://discuss.elastic.co/t/logstash-6-2-4-read-to-eof-no-delimiter-found-in-current-chunk/130613/5 "2018-05-04T11:26:45Z")
Let me open a new topic ---
**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\ **Post date:** [May 4, 2018, 11:27am UTC](https://discuss.elastic.co/t/logstash-6-2-4-read-to-eof-no-delimiter-found-in-current-chunk/130613/6 "2018-05-04T11:27:48Z")
No. Create a new topic only after you confirm that duplicates are seen with the new version. v4.1.2. ---
**Author:** ![ld\_pvl](https://avatars.discourse-cdn.com/v4/letter/l/cdc98d/32.png) [@ld\_pvl](https://discuss.elastic.co/u/ld_pvl)\ **Post date:** [May 4, 2018, 11:28am UTC](https://discuss.elastic.co/t/logstash-6-2-4-read-to-eof-no-delimiter-found-in-current-chunk/130613/7 "2018-05-04T11:28:12Z")
Ah, ok, roger. P.S.: my sincedb is not `/dev/null` and I've started from scratch. ---
**Author:** ![ld\_pvl](https://avatars.discourse-cdn.com/v4/letter/l/cdc98d/32.png) [@ld\_pvl](https://discuss.elastic.co/u/ld_pvl)\ **Post date:** [May 4, 2018, 1:57pm UTC](https://discuss.elastic.co/t/logstash-6-2-4-read-to-eof-no-delimiter-found-in-current-chunk/130613/8 "2018-05-04T13:57:18Z")
Related to the main topic - I'm getting this now after upgrading to 4.1.2. Should I take any action or is this some bug? ``` [2018-05-04T09:52:50,706][INFO][filewatch.tailmode.handlers.grow] buffer_extract: a delimiter can't be found in current chunk, maybe there are no more delimiters or the delimiter is incorrect or the text before the delimiter, a 'line', is very large, if this message is logged often try increasing the `file_chunk_size` setting. {"delimiter"=>"\n", "read_position"=>827326, "bytes_read_count"=>66, "last_known_file_size"=>827392, "file_path"=>"/foo/bar/my.log"} ``` FYI, I got this error while doing some testing in relation to this topic: [Logstash 6.2.1 Big .since-db file causes OutOfMemory](https://discuss.elastic.co/t/logstash-6-2-1-big-since-db-file-causes-outofmemory/129501) - and added a reply there. ---
**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\ **Post date:** [May 4, 2018, 2:41pm UTC](https://discuss.elastic.co/t/logstash-6-2-4-read-to-eof-no-delimiter-found-in-current-chunk/130613/9 "2018-05-04T14:41:20Z")
@ld_pvl This one is relevant and informative but whether it is important depends on the way the tailed files are being filled. I'll break it down. - `filewatch.tailmode.handlers.grow` - this is the code that executes when a file is seen to have grown from last time in tail mode. - `"delimiter"=>"\n"` - standard newline delimiter. - `"read_position"=>827326` - the offset in the file that the "chunk" was read from. - `"bytes_read_count"=>66` - the number of bytes in the "chunk" normally this is 32768 (32K). As it is smaller than 32K we are reading the last few bytes of the file **at the present time**. - `"last_known_file_size"=>827392` - the size of the file when we last checked (maybe 1 second, the `scan_interval` ago) 827326 + 66 = 827392 - yep, we read to the end of the file as we saw it at that time. Interpretation: 1. The system will be writing the rest of the line later. There was no newline character in those 66 bytes because the system writing the file may not have written/flushed the rest of the line and the closing newline yet. If this is the case then **we can ignore this message** , it is INFO after all. 2. The system will **not** be writing the rest of the line later. If the system never writes the rest of the line plus the newline then **this message is important**. It means that **those 66 bytes are lost** , they are stuck in the buffer (each file has its own buffer) and there is no more content to unstick it. Remember, that tailing is really an endless stream of content so there is no way of knowing that the buffer can be artificially flushed. Read mode to the rescue. In read mode, we assume that the file is a fixed length stream and in this case we can artificially flush the buffer when we reach the end of the stream. In 4.1.2 we have the limitation that discovered files should **not** be growing while (or after) we read the file, the additional content may not be read. This means that you should do an atomic write or copy - meaning write or copy to a folder outside of the `path` glob and then rename the file so it becomes discoverable. We have plans to fix this. Please tell me whether interpretation 1 or 2 applies to your case. ---
**Author:** ![ld\_pvl](https://avatars.discourse-cdn.com/v4/letter/l/cdc98d/32.png) [@ld\_pvl](https://discuss.elastic.co/u/ld_pvl)\ **Post date:** [May 6, 2018, 12:03am UTC](https://discuss.elastic.co/t/logstash-6-2-4-read-to-eof-no-delimiter-found-in-current-chunk/130613/10 "2018-05-06T00:03:45Z")
Thanks a lot ---
**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\ **Post date:** [May 9, 2018, 12:22pm UTC](https://discuss.elastic.co/t/logstash-6-2-4-read-to-eof-no-delimiter-found-in-current-chunk/130613/11 "2018-05-09T12:22:45Z")
See [this post for continuation](https://discuss.elastic.co/t/file-input-v4-1-2-tailing-duplicate-docs-ingested/130648/5?u=guyboertje). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 6, 2018, 12:22pm UTC](https://discuss.elastic.co/t/logstash-6-2-4-read-to-eof-no-delimiter-found-in-current-chunk/130613/12 "2018-06-06T12:22:51Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.