# Filebeat slowing to a halt within 20-30 minutes of starting

**URL:** <https://discuss.elastic.co/t/filebeat-slowing-to-a-halt-within-20-30-minutes-of-starting/83004>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [April 20, 2017, 8:02am UTC](https://discuss.elastic.co/t/filebeat-slowing-to-a-halt-within-20-30-minutes-of-starting/83004 "2017-04-20T08:02:07Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![thedongen](https://avatars.discourse-cdn.com/v4/letter/t/65b543/32.png) [@thedongen](https://discuss.elastic.co/u/thedongen)\
**Post date:** [April 20, 2017, 8:02am UTC](https://discuss.elastic.co/t/filebeat-slowing-to-a-halt-within-20-30-minutes-of-starting/83004/1 "2017-04-20T08:02:08Z")

</div>

I'm using the ELK stack to receive and parse logfiles from an Apache server. My Elasticsearch cluster consists of 3 nodes, with Logstash running on one of these nodes.

I've set up Filebeat on the machine that stores the logfiles as they're produced and have it pointing at my Logstash instance.

Previously, before using Filebeat I had no problems parsing logs directly from Logstash to Elasticsearch using the "file" input plugin, however, since beginning to use Filebeat, my indexing rate in Elasticsearch and events received/emitted rate in Logstash has plummeted.

This is a sample of what my events received and emitted over time looks like in Logstash (captured via X-Pack Monitoring):

 ![](https://us1.discourse-cdn.com/elastic/original/3X/f/d/fdeb294d1458c7d15ab7f5400cd5da9a955f7d56.PNG)

Logstash begins receiving an acceptable amount of throughput from Filebeat but quickly slows down to almost non-existant rates (and eventually stops). At the same time, event latency is also increasing.

For reference, I will include both my Logstash.yml and Filebeat.yml configuration.

Logstash.yml

```
path.data: /var/lib/logstash   
path.logs: /var/logs/logstash
path.settings: /etc/logstash

pipeline.workers: 4
pipeline.batch.size: 1000

xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.url: http://hostname:9200
xpack.monitoring.elasticsearch.username: "logstash_system"
xpack.monitoring.elasticsearch.password: "changeme"

log.level: info

```

Filebeat.yml

```
filebeat.spool_size: 4000
filebeat.idle_timeout: 5s

filebeat.prospectors:
- input_type: log
  paths:
     - /log/file/path/access_log-*

  close_inactive: 30s
  harvester_limit: 1024

output.logstash:
  hosts: ["hostname:5045"]
  bulk_max_size: 1000
  workers: 4

logging.level: debug
logging.selectors: ["*"]

```

I'm finding it tough to diagnose exactly what is causing the slowdown as the logs move from Filebeat \> Logstash \> Elasticsearch. Any and all help appreciated. I can also post any other configuration files that might be helpful.

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [April 20, 2017, 10:40am UTC](https://discuss.elastic.co/t/filebeat-slowing-to-a-halt-within-20-30-minutes-of-starting/83004/2 "2017-04-20T10:40:14Z")

</div>

can you share versions and logs?

---

<div class="post-metadata">

**Author:** ![thedongen](https://avatars.discourse-cdn.com/v4/letter/t/65b543/32.png) [@thedongen](https://discuss.elastic.co/u/thedongen)\
**Post date:** [April 20, 2017, 12:49pm UTC](https://discuss.elastic.co/t/filebeat-slowing-to-a-halt-within-20-30-minutes-of-starting/83004/3 "2017-04-20T12:49:35Z")

</div>

I'm running the latest version of each part of the stack, so 5.3 I believe?

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [April 21, 2017, 9:02am UTC](https://discuss.elastic.co/t/filebeat-slowing-to-a-halt-within-20-30-minutes-of-starting/83004/4 "2017-04-21T09:02:41Z")

</div>

And logs?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 19, 2017, 9:04am UTC](https://discuss.elastic.co/t/filebeat-slowing-to-a-halt-within-20-30-minutes-of-starting/83004/5 "2017-05-19T09:04:58Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.