# Packetbeat "dropping event, queue is blocked (seq=0)"

**URL:** <https://discuss.elastic.co/t/packetbeat-dropping-event-queue-is-blocked-seq-0/285219>\
**Category:** Beats\
**Tags:** packetbeat\
**Created:** [September 27, 2021, 10:18am UTC](https://discuss.elastic.co/t/packetbeat-dropping-event-queue-is-blocked-seq-0/285219 "2021-09-27T10:18:54Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![guido.lamoto](https://avatars.discourse-cdn.com/v4/letter/g/7c8e57/32.png) [@guido.lamoto](https://discuss.elastic.co/u/guido.lamoto)\
**Post date:** [September 27, 2021, 10:18am UTC](https://discuss.elastic.co/t/packetbeat-dropping-event-queue-is-blocked-seq-0/285219/1 "2021-09-27T10:18:54Z")

</div>

We have a Packetbeat 7.15.0 on Ubuntu 20.04 directly connected to a Debian 10 with Elasticsearch 7.14.1.

Packetbeat is running on a a DNS server with an average of 4000 queries/s. This is the configuration file:

```auto
packetbeat.interfaces.device: eno1
packetbeat.interfaces.type: af_packet
packetbeat.interfaces.snaplen: 1514
packetbeat.interfaces.buffer_size_mb: 100

packetbeat.protocols:
- type: dns
  ports: [53]

output.elasticsearch:
  hosts: ["192.168.255.250:9200"]

processors:
  - drop_fields:
      fields:
      [...]

monitoring.enabled: true

logging.to_files: true
logging.files:
  path: /var/log/packetbeat
  name: packetbeat

```

Packetbeat seems to drop (or to not emit) many events to Elasticsearch. These are the metrics:

 ![Screenshot at 2021-09-27 11-47-51](https://us1.discourse-cdn.com/elastic/original/3X/5/c/5c72b782c65d8dc66e70311b33de4f0e944ca0b7.png)

If we enable debug (filtered on publisher), we can see almost instantly the log full of:

> 2021-09-27T11:37:36.485+0200 DEBUG [publisher] memqueue/produce.go:150 Dropping event, queue is blocked (seq=0)

Seq number is always 0. No matter what, it seems that we cannot have more than ~900 new event published to Elasticsearch.

We have tried to change many options but the one that seems to have effects is **queue.mem.events**. If we set to _102400_ and restart Packetbeat we have a ~20 seconds of:

> 2021-09-27T11:37:23.258+0200 DEBUG [publisher] memqueue/ackloop.go:160 ackloop: receive ack [300: 0, 50]  
> 2021-09-27T11:37:23.258+0200 DEBUG [publisher] memqueue/ackloop.go:128 ackloop: return ack to broker loop:50  
> 2021-09-27T11:37:23.258+0200 DEBUG [publisher] memqueue/ackloop.go:131 ackloop: done send ack

After ~20 seconds (~350 ack received) we start to see again in the log:

> 2021-09-27T11:37:36.485+0200 DEBUG [publisher] memqueue/produce.go:150 Dropping event, queue is blocked (seq=0)  
> 2021-09-27T11:37:36.485+0200 DEBUG [publisher] pipeline/client.go:241 Pipeline client receives callback 'onDroppedOnPublish' for event: {...}

If we set queue.mem.events = _204800_ the ack are ~700 and then it starts again with the error about queue is blocked.

If we set queue.mem.events = _409600_ the ack are ~1500 and this is the situation:

 ![Screenshot at 2021-09-27 12-09-30](https://us1.discourse-cdn.com/elastic/original/3X/8/a/8a7a9fead93b0b17faa25e8c150845eb08fa45d8.png)

New events = queued events but still emitted events (and acknowledged events) are much \<\< than new events.

Where could be the problem? How can I debug such situation?

On a similar configuration (DNS server with Packetbeat + dedicated Elasticsearch server) we have this as expected:

 ![Screenshot at 2021-09-27 11-53-51](https://us1.discourse-cdn.com/elastic/original/3X/6/4/64980044914da4ac1bccceb3d39134113d9f2ba4.png)

All new events are emitted, acknowledged and queued.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 25, 2021, 12:19pm UTC](https://discuss.elastic.co/t/packetbeat-dropping-event-queue-is-blocked-seq-0/285219/2 "2021-10-25T12:19:04Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.