# Filebeat locks file when logstash is not runnig **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [November 21, 2016, 1:22pm UTC](https://discuss.elastic.co/t/filebeat-locks-file-when-logstash-is-not-runnig/66719 "2016-11-21T13:22:37Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![eluppol](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eluppol/32/13321_2.png) [@eluppol](https://discuss.elastic.co/u/eluppol)\ **Post date:** [November 21, 2016, 1:22pm UTC](https://discuss.elastic.co/t/filebeat-locks-file-when-logstash-is-not-runnig/66719/1 "2016-11-21T13:22:37Z")
I'm using filebeat to ship logs created by log4cxx to ELK stack. When filebeat is running but logstash is not available, log4cxx cannot rotate log file, so it becomes large (log generation speed is about 4 MB per minute, we use log rotation by 1 MB). I tried to set close\_timeout option to 1m, but filebeat still keeps the file open. Please see the log file generated by filebeat: > So how to make filebeat not block the file rotation? ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [November 21, 2016, 2:05pm UTC](https://discuss.elastic.co/t/filebeat-locks-file-when-logstash-is-not-runnig/66719/2 "2016-11-21T14:05:58Z")
I assume what you mean by `block file rotation` is keeping the file open. Because filebeat should not block any file rotation but will keep the file open until data is sent. As documented here [https://www.elastic.co/guide/en/beats/filebeat/5.0/configuration-filebeat-options.html#close-timeout](https://www.elastic.co/guide/en/beats/filebeat/5.0/configuration-filebeat-options.html#close-timeout) close\_timeout will not work if the output is completely blocked. What is your expect behaviour for filebeat related to the data? Should it just drop the data it did not send so far (means data loss)? ---
**Author:** ![eluppol](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eluppol/32/13321_2.png) [@eluppol](https://discuss.elastic.co/u/eluppol)\ **Post date:** [November 21, 2016, 6:29pm UTC](https://discuss.elastic.co/t/filebeat-locks-file-when-logstash-is-not-runnig/66719/3 "2016-11-21T18:29:25Z")
Yes, I expect that filebeat will drop the data which it cannot send. How to achieve this behavior? ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [November 22, 2016, 7:32am UTC](https://discuss.elastic.co/t/filebeat-locks-file-when-logstash-is-not-runnig/66719/4 "2016-11-22T07:32:38Z")
Currently there is no way to do this if your output is completely blocked. One "workaround" here is to use `harvester_limit` so not too much space is taken up. The disadvantage of this is that the files that are already open are kept and sent when the output gets available again, means some newer files will be dropped after rotation. The files that will be around will still start harvesting if the output comes available again. So I think in your case this is a feasible solution. ---
**Author:** ![eluppol](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eluppol/32/13321_2.png) [@eluppol](https://discuss.elastic.co/u/eluppol)\ **Post date:** [November 22, 2016, 1:03pm UTC](https://discuss.elastic.co/t/filebeat-locks-file-when-logstash-is-not-runnig/66719/5 "2016-11-22T13:03:32Z")
I tried harvester\_limit, but it didn't fix the problem. As I understood from filebeat manual, harvester\_limit just sets the maximum number of harvesters, but even the only harvester keeps file open, so it cannot be rotated. Also I noticed, that when filebeat is running and output is blocked, it locks only the last one file (log4cxx.log.10), so I cannot delete it by hand. Is there any other way to close files when output is blocked? Or maybe some way to use other output (for example to file) when logstash output is blocked? ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [November 22, 2016, 1:26pm UTC](https://discuss.elastic.co/t/filebeat-locks-file-when-logstash-is-not-runnig/66719/6 "2016-11-22T13:26:20Z")
The behaviour that you describe is what is expected with the harvester\_limit. I'm not sure why you mention that the file cannot be rotated? It should be possible to rename / rotate file, only deletion is not happening. So what you normally end up with is that you have several "deleted but open" files with the same name. What are you using for log rotation? ---
**Author:** ![eluppol](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eluppol/32/13321_2.png) [@eluppol](https://discuss.elastic.co/u/eluppol)\ **Post date:** [November 22, 2016, 1:53pm UTC](https://discuss.elastic.co/t/filebeat-locks-file-when-logstash-is-not-runnig/66719/7 "2016-11-22T13:53:28Z")
I'm using log4cxx RollingFileAppender configured to rotate files every 1MB. The problem is only with the last file, which cannot be deleted and as I understand it leads to situation when logs are accumulated in the original log file (log4cxx.log), so it becomes very large. When I stop filebeat, rotation process is resumed immediately, so log4cxx.log becomes log4cxx.log.1, log4cxx.log.1 becomes log4cxx.log.2 and so on... and log4cxx.log.10 is removed. ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [November 22, 2016, 2:41pm UTC](https://discuss.elastic.co/t/filebeat-locks-file-when-logstash-is-not-runnig/66719/8 "2016-11-22T14:41:16Z")
Seems to be the day of RollingFileAppender: [Log entries gets truncated on file rotation](https://discuss.elastic.co/t/log-entries-gets-truncated-on-file-rotation/66725/6?u=ruflin) Or is this other thread somehow related / same company? The part I still didn't understand about `RollingFileAppender` is when the initial file gets truncated / removed? ---
**Author:** ![eluppol](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eluppol/32/13321_2.png) [@eluppol](https://discuss.elastic.co/u/eluppol)\ **Post date:** [November 22, 2016, 3:38pm UTC](https://discuss.elastic.co/t/filebeat-locks-file-when-logstash-is-not-runnig/66719/9 "2016-11-22T15:38:44Z")
> [@ruflin](#): > > Seems to be the day of RollingFileAppender: [Log entries gets truncated on file rotation - #6 by ruflin](https://discuss.elastic.co/t/log-entries-gets-truncated-on-file-rotation/66725/6) Or is this other thread somehow related / same company? Yeah, but I don't think this thread is somehow related 🙂 > [@ruflin](#): > > The part I still didn't understand about RollingFileAppender is when the initial file gets truncated / removed? We use RollingFileAppender with MaxBackupIndex set to 10, so it creates only 10 files during log rotation. As I understand, RollingFileAppender just removes the last file, renames the initial file and all the subsequent files, and then new initial file is created. As an example I tried to configure filebeat to read only from initial file. The rotation process stalled when initial file became the last one and I think RollingFileAppender couldn't remove it because it was opened by filebeat. ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [November 23, 2016, 10:59am UTC](https://discuss.elastic.co/t/filebeat-locks-file-when-logstash-is-not-runnig/66719/10 "2016-11-23T10:59:42Z")
Ok, that sounds like "normal" log rotation. What is your operating system / file system? I see some Windows path in your logs? This could be a Windows related issue ☹ ---
**Author:** ![eluppol](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eluppol/32/13321_2.png) [@eluppol](https://discuss.elastic.co/u/eluppol)\ **Post date:** [November 23, 2016, 11:22am UTC](https://discuss.elastic.co/t/filebeat-locks-file-when-logstash-is-not-runnig/66719/11 "2016-11-23T11:22:16Z")
We're using Windows Server 2008 R2 Enterprise and NTFS file system. ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [November 25, 2016, 10:24am UTC](https://discuss.elastic.co/t/filebeat-locks-file-when-logstash-is-not-runnig/66719/12 "2016-11-25T10:24:27Z")
Do you get any errors from log4j when it tries to rotate / remove the file? I currently assume on Windows deleted but still open files can't have the same name (possible on Linux). As the file is not deleted, log rotation is actually blocked. So instead of having open file handlers on deleted files like on windows, the initial file written to is constantly growing. Here is a github issue where we are discussing about a config option like `drop_events` or similar which would solve this issue: [https://github.com/elastic/beats/issues/2395](https://github.com/elastic/beats/issues/2395) But it would mean losing events. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [December 23, 2016, 10:24am UTC](https://discuss.elastic.co/t/filebeat-locks-file-when-logstash-is-not-runnig/66719/13 "2016-12-23T10:24:44Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.