# Missing a lot of logs sent via filebeats

**URL:** <https://discuss.elastic.co/t/missing-a-lot-of-logs-sent-via-filebeats/61185>\
**Category:** Beats\
**Created:** [September 21, 2016, 9:41pm UTC](https://discuss.elastic.co/t/missing-a-lot-of-logs-sent-via-filebeats/61185 "2016-09-21T21:41:35Z")\
**Posts on this page:** 1\
**Showing post:** 5

<div class="post-metadata">

**Author:** ![il.bert](https://avatars.discourse-cdn.com/v4/letter/i/ac91a4/32.png) [@il.bert](https://discuss.elastic.co/u/il.bert)\
**Post date:** [September 22, 2016, 8:23am UTC](https://discuss.elastic.co/t/missing-a-lot-of-logs-sent-via-filebeats/61185/5 "2016-09-22T08:23:23Z")

</div>

LS I used both 2.4 and 5.0  
FB is 5alpha6

FB is the following

for each file I have this path conf

```
  fields_under_root: true
  ignore_older: 10m
  close_inactive: 2m
  clean_inactive: 15m

#========================= Filebeat global options ============================

filebeat.spool_size: 10000
filebeat.idle_timeout: 10s

#----------------------------- Logstash output --------------------------------
output.logstash:
  hosts: ["10.246.85.242:5044", "10.246.85.243:5044"]
  template.name: "filebeat"
  loadbalance: true
  bulk_max_size: 1000  
  template.path: "filebeat.template.json"
  template.overwrite: false

```

in LS i tried LOTS OF configuration 2, 3 and 4 instances of LS with different conf  
the one that get me better results is the following, different configurations allowed me to pass a very smaller throughput (10% of the load) with this one I have almost 75%

```
path.data: /var/lib/logstash
pipeline.workers: 24
pipeline.output.workers: 1
pipeline.batch.size: 15000
path.config: /etc/logstash/conf.d
config.reload.automatic: true
config.reload.interval: 30
log.level: verbose
path.log: /var/log/logstash/logstash.log

```

with

```
-Xms12g
-Xmx12g

```

I have opened also this [FileBeat EOF Error](https://discuss.elastic.co/t/filebeat-eof-error/60882/5) and this [FileBeat file is falling under ignore older](https://discuss.elastic.co/t/filebeat-file-is-falling-under-ignore-older/61210) for other errors i am facing

hope you will help asap as my POC for my manager should be ready by next week

---

_[View the full topic](https://discuss.elastic.co/t/missing-a-lot-of-logs-sent-via-filebeats/61185)._