# Inactive and Rolling files

**URL:** https://discuss.elastic.co/t/inactive-and-rolling-files/161398
**Category:** Beats
**Tags:** filebeat
**Created:** [December 18, 2018, 6:49pm UTC](https://discuss.elastic.co/t/inactive-and-rolling-files/161398 "2018-12-18T18:49:20Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![mickt](https://avatars.discourse-cdn.com/v4/letter/m/aca169/32.png) [@mickt](https://discuss.elastic.co/u/mickt)
#### Post date: [December 18, 2018, 6:49pm UTC](https://discuss.elastic.co/t/inactive-and-rolling-files/161398/1 "2018-12-18T18:49:20Z")

</div>

Hi,

I have a question regarding the close\_inactive status and rolling files configuration of Filebeat.

I currently have Filebeat shipping some apache logs.  
Apache Log entries can come in every 2 seconds as well as every 48 hours (This is a test environment that is not heavily used).

There is a rolling mechanism gzipping access\_log file every day.

Currently the Filebeat configuration is using the default 5m close\_inactive settings.

```
filebeat.inputs:
- type: log
  paths:
    - "/var/log/httpd/access_log"
  fields:
    log:
      application: apache2
      type: access
  fields_under_root: true

```

So when I start the beat it ships apache logs to logstash.  
After 5m of Apache inactivity on the access\_log I get  
`File is inactive: /var/log/httpd/access_log. Closing because close_inactive of 5m0s reached`  
and Filebeat stops sending log updates to logstash after that.

New logs arriving in access\_log are not shipped anymore.  
So after 5 minutes of inactivity of Apache I am left with a Filebeat closing the file and not shipping logs thereafter.

> If a file is updated after the harvester is closed, the file will be picked up again after `scan_frequency` has elapsed.

> scan\_frequency  
> If you require log lines to be sent in near real time do not use a very low `scan_frequency` but adjust `close_inactive` so the file handler stays open and constantly polls your files.  
> The default setting is 10s.

I understood that _scan\_frequency_ would be executed by default every 10s making Filebeat monitoring access\_log new entries even after the file has been closed for inactivity.  
This is actually not happening. After close\_inactive time has passed new logs are never picked up again.

The other doubt I have is related to Filebeat file handing when a file gets rolled (gzipped).  
Is there something I need to set in order to have Filebeat keep track of the new /var/log/httpd/access\_log once the rolling operation is triggered?  
It seems that when the file is gzipped I am also loosing Filebeat track of it.

Thanks

---

<div class="post-metadata">

### Author: ![mickt](https://avatars.discourse-cdn.com/v4/letter/m/aca169/32.png) [@mickt](https://discuss.elastic.co/u/mickt)
#### Post date: [December 20, 2018, 11:01pm UTC](https://discuss.elastic.co/t/inactive-and-rolling-files/161398/2 "2018-12-20T23:01:18Z")

</div>

scan\_frequency seems to behave correctly on my local box running Docker Engine Community **18.09.0**  
I start thinking it might be a problem with the Docker version installed on the server **1.13.1**

Docker version is the only difference I see (or maybe something at docker config level).  
'mick' user is actually part of docker group on both systems

I have noticed this

```
mick@local:/tmp $ docker run -v /tmp:/tmp alpine touch /tmp/test.txt
mick@local:/tmp $ ls -lart
-rw-r--r-- 1 mick mick 0B Dec 20 23:50 test.txt

```

while on the Server the test.txt results own by root

```
mick@server:/tmp $ docker run -v /tmp:/tmp alpine touch /tmp/test.txt
mick@server:/tmp $ ls -lart
-rw-r--r-- 1 root root 0B Dec 20 23:50 test.txt

```

Not sure why is this happening.  
Could this cause problems with Filebeat scan\_frequency configuration?

As I said, on the Server once the File has been closed, Filebeats will never scan it again for updates and it stays at

> "filebeat":{"harvester":{"open\_files":0,"running":0}}

On the local box if the logs receives new updates it gets reopened.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 17, 2019, 11:01pm UTC](https://discuss.elastic.co/t/inactive-and-rolling-files/161398/3 "2019-01-17T23:01:29Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
