# Dropped Netflow packets in filebeat

**URL:** <https://discuss.elastic.co/t/dropped-netflow-packets-in-filebeat/283837>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [September 10, 2021, 3:07am UTC](https://discuss.elastic.co/t/dropped-netflow-packets-in-filebeat/283837 "2021-09-10T03:07:39Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![hjazz6](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hjazz6/32/79007_2.png) [@hjazz6](https://discuss.elastic.co/u/hjazz6)\
**Post date:** [September 10, 2021, 3:07am UTC](https://discuss.elastic.co/t/dropped-netflow-packets-in-filebeat/283837/1 "2021-09-10T03:07:39Z")

</div>

Hi,

I'm ingesting Netflow traffic using filebeat's netflow module (for the first time), and I think there are dropped packets. I'm wondering if there is anything I can do to reduce or eliminate dropped packets.

I started filebeat using `filebeat -e` so I can see the stats on my screen. The final stats are as follows.

```auto
"filebeat": {
   "events": {
      "added": 430866,
      "done": 430866
   },
   "harvester": {
      "open_files": 0,
      "running": 0
   },
   "input": {
      "netflow": {
         "flows": 430866,
         "packets": {
            "dropped": 854349,
            "received": 710962
         }
      }
   }
}

```

I'm assuming since `dropped` is non-zero, I'm losing some of my netflow traffic. I also see `libbeat.pipeline.queue.max_events=4096`. I've set `var.queue_size` to `8192` in `netflow.yml`, but I still see `libbeat.pipeline.queue.max_events=4096` in the stats after restarting filebeat.

My server has 48 cores and 125GB of memory. I've set my heap memory to 64GB for ElasticSearch (not sure if that helps). I'm also using HDD instead of SSD, which I know limits my IO performance.

What other things can I do to minimize the dropped packets?

Thank you!

---

<div class="post-metadata">

**Author:** ![techie.antonio](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/techie.antonio/32/77423_2.png) [@techie.antonio](https://discuss.elastic.co/u/techie.antonio)\
**Post date:** [October 6, 2021, 10:28am UTC](https://discuss.elastic.co/t/dropped-netflow-packets-in-filebeat/283837/2 "2021-10-06T10:28:08Z")

</div>

Using HDDs is most like a limiter on the Elasticsearch side. We use ElastiFlow - [the new one](https://docs.elastiflow.com/docs/) - and the guy that created it has a video that shows how bad HDDs are for Elasticsearch.

[![](https://us1.discourse-cdn.com/elastic/original/3X/6/3/634a9d322a83091765b945bdac29d4ee085c4c50.jpeg "What is the best storage technology for Elasticsearch?") ](https://www.youtube.com/watch?v=nKUpfJCBiS4)

We also tried filebeat, but ElastiFlow had much better throughput and more features.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 3, 2021, 12:28pm UTC](https://discuss.elastic.co/t/dropped-netflow-packets-in-filebeat/283837/3 "2021-11-03T12:28:53Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.