# Filebeat stop to harvest for nginx log \[Solved\]

**URL:** <https://discuss.elastic.co/t/filebeat-stop-to-harvest-for-nginx-log-solved/169489>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [February 21, 2019, 8:53pm UTC](https://discuss.elastic.co/t/filebeat-stop-to-harvest-for-nginx-log-solved/169489 "2019-02-21T20:53:55Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Lucio\_Palmieri](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lucio_palmieri/32/41152_2.png) [@Lucio\_Palmieri](https://discuss.elastic.co/u/Lucio_Palmieri)\
**Post date:** [February 21, 2019, 8:53pm UTC](https://discuss.elastic.co/t/filebeat-stop-to-harvest-for-nginx-log-solved/169489/1 "2019-02-21T20:53:55Z")

</div>

Hi everyone,

I'm trying to use ELK to analyze Nginx access.log, but when I run Filebeat, I get a few hours of logging until Filebeat enters in an infinite loop where it says:  
"Harvester for file is still running ..."

## Filebeat version

`$ filebeat version`

```
filebeat version 6.6.0 (amd64), libbeat 6.6.0

```

## Operating System

`$ cat /etc/issue`

```
Ubuntu 16.04.4 LTS

```

## Configuration

`$ sudo cat /etc/filebeat/filebeat.yml`

```
filebeat.inputs:
- type: log
  ignore_older: 48
  clean_inactive: 72
  clean_removed: true

  enabled: true

  paths:
    - /var/log/nginx/access.log

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false

setup.template.settings:
  index.number_of_shards: 3

setup.dashboards.enabled: false

setup.kibana:
  host: "10.10.0.134:8080"

output.logstash:
  hosts: ["10.10.0.128:5044"]

processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~

```

`$ sudo cat /etc/filebeat/modules.d/nginx.yml`

```
- module: nginx
  access:
    enabled: true
  error:
    enabled: false

```

## Nginx logrotate

`$ sudo cat /etc/logrotate.d/nginx`

```
/var/log/nginx/*.log {
  daily
  missingok
  rotate 14
  compress
  delaycompress
  notifempty
  create 0640 www-data adm
  sharedscripts
  prerotate
    if [-d /etc/logrotate.d/httpd-prerotate]; then \
      run-parts /etc/logrotate.d/httpd-prerotate; \
    fi \
  endscript
  postrotate
    invoke-rc.d nginx rotate >/dev/null 2>&1
  endscript
}

```

## Debug info

`$ sudo filebeat -e -d "*" -c /etc/filebeat/filebeat.yml -path.home /usr/share/filebeat -path.config /etc/filebeat -path.data /var/lib/filebeat`

```
...
    "containerized": false,
    "name": "ip-10-10-2-214",
    "architecture": "x86_64",
    "os": {
      "family": "debian",
      "name": "Ubuntu",
      "codename": "xenial",
      "platform": "ubuntu",
      "version": "16.04.4 LTS (Xenial Xerus)"
    }
  },
  "beat": {
    "name": "ip-10-10-2-214",
    "hostname": "ip-10-10-2-214",
    "version": "6.6.0"
  },
  "source": "/var/log/nginx/access.log",
  "offset": 529532
}
DEBUG	[input]	input/input.go:152	Run input
DEBUG	[input]	log/input.go:174	Start next scan
DEBUG	[input]	log/input.go:404	Check file for harvesting: /var/log/nginx/access.log
DEBUG	[input]	log/input.go:494	Update existing file for harvesting: /var/log/nginx/access.log, offset: 529645
DEBUG	[input]	log/input.go:546	Harvester for file is still running: /var/log/nginx/access.log
DEBUG	[input]	log/input.go:195	input states cleaned up. Before: 1, After: 1, Pending: 1
DEBUG	[input]	input/input.go:152	Run input
DEBUG	[input]	log/input.go:174	Start next scan
DEBUG	[input]	log/input.go:255	Exclude file: /var/log/nginx/access.log.10.gz
                                    ...
DEBUG	[input]	log/input.go:255	Exclude file: /var/log/nginx/access.log.9.gz
DEBUG	[input]	log/input.go:404	Check file for harvesting: /var/log/nginx/access.log.1
DEBUG	[input]	log/input.go:494	Update existing file for harvesting: /var/log/nginx/access.log.1, offset: 245557
DEBUG	[input]	log/input.go:546	Harvester for file is still running: /var/log/nginx/access.log.1
DEBUG	[input]	log/input.go:404	Check file for harvesting: /var/log/nginx/access.log
DEBUG	[input]	log/input.go:494	Update existing file for harvesting: /var/log/nginx/access.log, offset: 263515
DEBUG	[input]	log/input.go:546	Harvester for file is still running: /var/log/nginx/access.log
DEBUG	[input]	log/input.go:195	input states cleaned up. Before: 2, After: 2, Pending: 0
DEBUG	[input]	input/input.go:152	Run input
DEBUG	[input]	log/input.go:174	Start next scan
DEBUG	[input]	log/input.go:404	Check file for harvesting: /var/log/nginx/access.log
DEBUG	[input]	log/input.go:494	Update existing file for harvesting: /var/log/nginx/access.log, offset: 529645
DEBUG	[input]	log/input.go:546	Harvester for file is still running: /var/log/nginx/access.log
DEBUG	[input]	log/input.go:195	input states cleaned up. Before: 1, After: 1, Pending: 1
DEBUG	[input]	input/input.go:152	Run input
DEBUG	[input]	log/input.go:174	Start next scan
DEBUG	[input]	log/input.go:255	Exclude file: /var/log/nginx/access.log.10.gz
                                    ...
DEBUG	[input]	log/input.go:255	Exclude file: /var/log/nginx/access.log.9.gz
DEBUG	[input]	log/input.go:404	Check file for harvesting: /var/log/nginx/access.log
DEBUG	[input]	log/input.go:494	Update existing file for harvesting: /var/log/nginx/access.log, offset: 263515
DEBUG	[input]	log/input.go:546	Harvester for file is still running: /var/log/nginx/access.log
DEBUG	[input]	log/input.go:404	Check file for harvesting: /var/log/nginx/access.log.1
DEBUG	[input]	log/input.go:494	Update existing file for harvesting: /var/log/nginx/access.log.1, offset: 245557
DEBUG	[input]	log/input.go:546	Harvester for file is still running: /var/log/nginx/access.log.1
DEBUG	[input]	log/input.go:195	input states cleaned up. Before: 2, After: 2, Pending: 0
INFO	[monitoring]	log/log.go:144	Non-zero metrics in the last 30s	{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":400,"time": {"ms":400}},"total":{"ticks":1010,"time":{"ms":1016},"value":1010},"user":{"ticks":610,"time": {"ms":616}}},"handles":{"limit":{"hard":1048576,"soft":1024},"open":9},"info": {"ephemeral_id":"0ad99bff-ef64-4b7a-b488-db0c3635912a","uptime": {"ms":30013}},"memstats":{"gc_next":36515312,"memory_alloc":29067496,"memory_total":82606960,"rss":59797504}},"fil ebeat":{"events":{"active":4120,"added":4125,"done":5},"harvester":    
{"open_files":3,"running":3,"started":3}},"libbeat":{"config":{"module":    
{"running":0},"reloads":1},"output":{"events":{"active":2104,"batches":3,"total":2104},"read":    
{"bytes":30},"type":"logstash","write":{"bytes":244874}},"pipeline":{"clients":3,"events":    
{"active":4118,"filtered":6,"published":4116,"retry":1025,"total":4124}}},"registrar":{"states":        
{"current":1,"update":5},"writes":{"success":5,"total":5}},"system":{"cpu":{"cores":16},"load":    
{"1":4.35,"15":4.28,"5":4.44,"norm":{"1":0.2719,"15":0.2675,"5":0.2775}}}}}}
DEBUG	[input]	input/input.go:152	Run input
...

```

So it happens that I receive a few hours of data from the log

![05](https://us1.discourse-cdn.com/elastic/original/3X/a/3/a3cc7d2b99477f2a7545a72e2bfd54fad8571cc1.png)

then, in the filebeat log, an infinite loop starts in which I see the messages repeat:

DEBUG [input] log / input.go: 546 **Harvester for file is still running** : /var/log/nginx/access.log  
DEBUG [input] log / input.go: 195 input states cleaned up. Before: 2, After: 2, Pending: 0

I hope to receive some suggestions to solve this problem.  
**Thanks in advance**  
Lucio

---

<div class="post-metadata">

**Author:** ![Lucio\_Palmieri](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lucio_palmieri/32/41152_2.png) [@Lucio\_Palmieri](https://discuss.elastic.co/u/Lucio_Palmieri)\
**Post date:** [February 22, 2019, 9:22pm UTC](https://discuss.elastic.co/t/filebeat-stop-to-harvest-for-nginx-log-solved/169489/2 "2019-02-22T21:22:44Z")

</div>

Hi everyone,  
finally I solved and I share the solution hoping it can be useful to others.  
Reading [filebeat docs](https://github.com/elastic/beats/blob/99824df163863478d27688206e18ec27bc1b3ac5/filebeat/docs/how-filebeat-works.asciidoc) , I realized that the problem could be in **logstash** and then I also analyzed its log:

```
FORBIDDEN/12/index read-only / allow delete (api)]

```

so I found the answer in this [Topic](https://discuss.elastic.co/t/forbidden-12-index-read-only-allow-delete-api/110282/4):

> i think my problem is **low storage**. just check your storage first. when it's low, kibana auto changes its config to read-only mode.

So I solved my problem by increasing disk space on the elasticsearch host and removing the read-only mode on the indices by running in kibana:

```
PUT _settings
{
  "index": {
    "blocks": {
      "read_only_allow_delete": "false"
    }
  }
}

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 22, 2019, 9:22pm UTC](https://discuss.elastic.co/t/filebeat-stop-to-harvest-for-nginx-log-solved/169489/3 "2019-03-22T21:22:56Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.