# Some lines sent to Logstash are truncated **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [March 24, 2017, 2:22pm UTC](https://discuss.elastic.co/t/some-lines-sent-to-logstash-are-truncated/79900 "2017-03-24T14:22:34Z")\ **Posts on this page:** 16\ **Page:** 1
**Author:** ![Pierre\_Vincent\_Ledou](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierre_vincent_ledou/32/1368_2.png) [@Pierre\_Vincent\_Ledou](https://discuss.elastic.co/u/Pierre_Vincent_Ledou)\ **Post date:** [March 24, 2017, 2:22pm UTC](https://discuss.elastic.co/t/some-lines-sent-to-logstash-are-truncated/79900/1 "2017-03-24T14:22:34Z")
Hi, I'm parsing a lot of old logs files. All logs are in gz, so I have to uncompress then move it a folder watched by filebeat. On about 30millions of entries, I have about 1300 failures in logstash logs. I'm logging the messages so I can see that Logstash received a partial line, the line is truncated randomly. I doubled check to ensure that I don't have any special characters or so in my logs. So why Filebeat is sending partial lines? ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [March 27, 2017, 8:16am UTC](https://discuss.elastic.co/t/some-lines-sent-to-logstash-are-truncated/79900/2 "2017-03-27T08:16:04Z")
Could you share the following? - Filebeat configs - Filebeat logs - Filebeat version - Logstash config - An example of a line that was "partial" ---
**Author:** ![Pierre\_Vincent\_Ledou](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierre_vincent_ledou/32/1368_2.png) [@Pierre\_Vincent\_Ledou](https://discuss.elastic.co/u/Pierre_Vincent_Ledou)\ **Post date:** [March 27, 2017, 8:26am UTC](https://discuss.elastic.co/t/some-lines-sent-to-logstash-are-truncated/79900/3 "2017-03-27T08:26:42Z")
Yes no problem, but I would prefer to send it in MP if it's ok for you? ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [March 28, 2017, 2:09pm UTC](https://discuss.elastic.co/t/some-lines-sent-to-logstash-are-truncated/79900/4 "2017-03-28T14:09:42Z")
You mean the logs? That is ok for me. For the other files it should be possible to post them here (but remove passwords 😉 ). ---
**Author:** ![Pierre\_Vincent\_Ledou](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierre_vincent_ledou/32/1368_2.png) [@Pierre\_Vincent\_Ledou](https://discuss.elastic.co/u/Pierre_Vincent_Ledou)\ **Post date:** [April 19, 2017, 7:44am UTC](https://discuss.elastic.co/t/some-lines-sent-to-logstash-are-truncated/79900/5 "2017-04-19T07:44:43Z")
Ops, sorry for the delay, I missed your reply. filebeat.yml ``` filebeat: prospectors: - paths: - /var/data/level3/beats/*.log document_type: level3_log exclude_lines: ['^#'] close_inactive: 10s registry_file: /var/data/filebeat_registry logging.level: info logging.metrics.enabled: false logging.to_files: false logging.to_syslog: false output: logstash: hosts: ["logstash:5044"] ``` logstash.yml ``` config.reload.automatic: true config.reload.interval: 5 queue.type: persisted path.queue: /usr/share/logstash/queue path.logs: /usr/share/logstash/log pipeline.workers: 8 pipeline.batch.size: 2500 pipeline.batch.delay: 5 http.host: "0.0.0.0" xpack.monitoring.enabled: true xpack.monitoring.elasticsearch.url: http://elasticsearch:9200 ``` The logstash filters and groks are quite heavy, I'm zipping it in MP right now with some logs samples. Thanks for you help! Cheers, Pv ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [April 24, 2017, 7:15am UTC](https://discuss.elastic.co/t/some-lines-sent-to-logstash-are-truncated/79900/6 "2017-04-24T07:15:41Z")
Thanks for the data. Could you provide an example message which was truncated? Also I was looking for the Filebeat logs. Do you see anything special in there? Is the volume you read logs from a shared drive and somehow mounted or a local disk? If you write the log output to file instead of LS, do you still see it happening? ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [April 24, 2017, 7:16am UTC](https://discuss.elastic.co/t/some-lines-sent-to-logstash-are-truncated/79900/7 "2017-04-24T07:16:28Z")
BTW: Which filebeat, logstash, logstash-beats-input version are you running? ---
**Author:** ![Pierre\_Vincent\_Ledou](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierre_vincent_ledou/32/1368_2.png) [@Pierre\_Vincent\_Ledou](https://discuss.elastic.co/u/Pierre_Vincent_Ledou)\ **Post date:** [April 25, 2017, 7:42am UTC](https://discuss.elastic.co/t/some-lines-sent-to-logstash-are-truncated/79900/8 "2017-04-25T07:42:23Z")
I'm now on 5.3 for all the stack except Filebeat that is still in 5.2.2. I'm running filebeat on a unique node as a docker container but this issue was already existing when filebeat was running directly on the host). The disk is not a ssd but a raid 5 sata. Filebeat is streaming to 3 Logstash nodes (a container on the same node and 2 others remote). I will try to make some test to write on disk directly today or tomorrow. I send you the failure log in MP. ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [April 26, 2017, 12:59pm UTC](https://discuss.elastic.co/t/some-lines-sent-to-logstash-are-truncated/79900/9 "2017-04-26T12:59:40Z")
Quite often such behaviour can come from shared drives, but inside docker when on Linux should be ok. Other ideas: - How do you remove the files after you index them? - Could it be that you some inode reuse issue? When do you remove old files? ---
**Author:** ![Pierre\_Vincent\_Ledou](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierre_vincent_ledou/32/1368_2.png) [@Pierre\_Vincent\_Ledou](https://discuss.elastic.co/u/Pierre_Vincent_Ledou)\ **Post date:** [April 26, 2017, 3:01pm UTC](https://discuss.elastic.co/t/some-lines-sent-to-logstash-are-truncated/79900/10 "2017-04-26T15:01:37Z")
> [@ruflin](#): > > - How do you remove the files after you index them? I have script reading the registry. If the offset = file size, I delete it > [@ruflin](#): > > - Could it be that you some inode reuse issue? When do you remove old files? I have `close_inactive: 10s` in the config, and my script is running every minute. Cheers, Pv ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [April 28, 2017, 8:02am UTC](https://discuss.elastic.co/t/some-lines-sent-to-logstash-are-truncated/79900/11 "2017-04-28T08:02:37Z")
Could it be that your partial lines come actually from an other file because it reuses the inode? We had a similar case here: [https://github.com/elastic/beats/issues/714#issuecomment-295329605](https://github.com/elastic/beats/issues/714#issuecomment-295329605) If that is the case, I recommend you to first move the files to an other place to clean up the registry and then remove the files later. This will prevent the inode reuse. ---
**Author:** ![Pierre\_Vincent\_Ledou](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierre_vincent_ledou/32/1368_2.png) [@Pierre\_Vincent\_Ledou](https://discuss.elastic.co/u/Pierre_Vincent_Ledou)\ **Post date:** [May 19, 2017, 2:01pm UTC](https://discuss.elastic.co/t/some-lines-sent-to-logstash-are-truncated/79900/12 "2017-05-19T14:01:29Z")
Hi, sorry for the late answer, I was waiting to be sure that the issue was resolved. Now I'm moving finished logs to a tmp dir instead of delete them, and I think that solved my problem. Instead of inode, wouldn't possible to use file path? Or make it configurable for user like me that parse logs not in real time? ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [May 22, 2017, 10:36am UTC](https://discuss.elastic.co/t/some-lines-sent-to-logstash-are-truncated/79900/13 "2017-05-22T10:36:15Z")
Glad that solve the problem. About using path as identifier instead of inode: Agree. This should be an option to configure or even be a separate prospector type for example file where it is assumed that files are never renamed or data is never appended. Feel free to open a feature request for this on Github. ---
**Author:** ![Pierre\_Vincent\_Ledou](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierre_vincent_ledou/32/1368_2.png) [@Pierre\_Vincent\_Ledou](https://discuss.elastic.co/u/Pierre_Vincent_Ledou)\ **Post date:** [May 22, 2017, 11:39am UTC](https://discuss.elastic.co/t/some-lines-sent-to-logstash-are-truncated/79900/14 "2017-05-22T11:39:11Z")
I will 😉 Thanks a lot! ---
**Author:** ![Pierre\_Vincent\_Ledou](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierre_vincent_ledou/32/1368_2.png) [@Pierre\_Vincent\_Ledou](https://discuss.elastic.co/u/Pierre_Vincent_Ledou)\ **Post date:** [May 22, 2017, 11:51am UTC](https://discuss.elastic.co/t/some-lines-sent-to-logstash-are-truncated/79900/15 "2017-05-22T11:51:52Z")
Done: [https://github.com/elastic/beats/issues/4368](https://github.com/elastic/beats/issues/4368) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 19, 2017, 11:52am UTC](https://discuss.elastic.co/t/some-lines-sent-to-logstash-are-truncated/79900/16 "2017-06-19T11:52:23Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.