# Filebeat handles rotated files issue

**URL:** https://discuss.elastic.co/t/filebeat-handles-rotated-files-issue/152298
**Category:** Beats
**Tags:** filebeat
**Created:** [October 12, 2018, 8:17pm UTC](https://discuss.elastic.co/t/filebeat-handles-rotated-files-issue/152298 "2018-10-12T20:17:39Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![IngZero2](https://avatars.discourse-cdn.com/v4/letter/i/e95f7d/32.png) [@IngZero2](https://discuss.elastic.co/u/IngZero2)
#### Post date: [October 12, 2018, 8:17pm UTC](https://discuss.elastic.co/t/filebeat-handles-rotated-files-issue/152298/1 "2018-10-12T20:17:39Z")

</div>

Hi all,  
i've installed Filebeat on a AKS cluster with 5 nodes.  
I've a stranger behaviour with rotated containers log files.  
Docker rotation rule is  
`{ "live-restore": true, "log-driver": "json-file", "log-opts": { "max-size": "50m", "max-file": "5" } }`  
When some container logs very often, the rotation is quickly. In this case Filebeat doesn't release the file handler and the result is the disk is full very fast.  
I tried to use the close\_timeout option, but is not enough.  
How i can bypass this issue?

filebeat config map :  
`filebeat.inputs:

- type: log  
paths:
  - /var/log/containers/\*.log  
scan\_frequency: 10s  
close\_timeout: 5h  
symlinks: true  
json.message\_key: log  
json.keys\_under\_root: true  
json.add\_error\_key: true  
multiline.pattern: '^\s'  
multiline.match: after  
fields\_under\_root: true  
fields:  
type: kube-logs  
node.hostname: ${NODE\_HOSTNAME}  
pod.ip: ${POD\_IP}  
tags:
  - k8s-app  
filebeat.config.modules:

# Set to true to enable config reloading
reload.enabled: true  
output.logstash:  
hosts: logging-logstash:5000  
timeout: 60  
bulk\_max\_size: 1024  
logging.level: info  
logging.selectors: [logstash]`

Thanks

---

<div class="post-metadata">

### Author: ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [October 16, 2018, 8:51pm UTC](https://discuss.elastic.co/t/filebeat-handles-rotated-files-issue/152298/2 "2018-10-16T20:51:06Z")

</div>

The logs are written faster then logstash accepts them. Normally Filebeat keeps the files open until end of file. You have configured `close_timeout: 5h`. This is a very big value. Using a much lower value can force filebeat to close files much faster.

---

<div class="post-metadata">

### Author: ![IngZero2](https://avatars.discourse-cdn.com/v4/letter/i/e95f7d/32.png) [@IngZero2](https://discuss.elastic.co/u/IngZero2)
#### Post date: [October 17, 2018, 4:33pm UTC](https://discuss.elastic.co/t/filebeat-handles-rotated-files-issue/152298/3 "2018-10-17T16:33:01Z")

</div>

Thanks,  
i reduced the close\_timeout to 1h, it's enough os should be less then 1h?  
close\_timeout force to close all the harvester?

---

<div class="post-metadata">

### Author: ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [October 18, 2018, 8:56pm UTC](https://discuss.elastic.co/t/filebeat-handles-rotated-files-issue/152298/4 "2018-10-18T20:56:21Z")

</div>

How fast do you rotate? Some users set it even to 1min.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 15, 2018, 9:09pm UTC](https://discuss.elastic.co/t/filebeat-handles-rotated-files-issue/152298/5 "2018-11-15T21:09:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
