# Pipeline with Persistent Queues stops working after some hours

**URL:** <https://discuss.elastic.co/t/pipeline-with-persistent-queues-stops-working-after-some-hours/121841>\
**Category:** Logstash\
**Created:** [February 28, 2018, 11:31am UTC](https://discuss.elastic.co/t/pipeline-with-persistent-queues-stops-working-after-some-hours/121841 "2018-02-28T11:31:26Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![gghdr](https://avatars.discourse-cdn.com/v4/letter/g/e19adc/32.png) [@gghdr](https://discuss.elastic.co/u/gghdr)\
**Post date:** [February 28, 2018, 11:31am UTC](https://discuss.elastic.co/t/pipeline-with-persistent-queues-stops-working-after-some-hours/121841/1 "2018-02-28T11:31:26Z")

</div>

Hello,

i have a pipeline with the following plugins:  
input: Kafka,Heartbeat  
filters: Metric,Sleep,Clone,Mutate  
outputs: File,Kafka,Jms (Websphere MQ Queue)

after working fine for some hours, **logstash stops to process events**. The Heartbeat stops too. If i enter in DEBUG mode i still see "Pushing flush onto pipeline" but no events are processed. If i restart it works fine again for some hours until it stops again.

I tryed to change settings inside jvm.options and logstash.yml files without success.

Someone has already sees this behaviour? what's really strange to me is that Heartbeat stops printing its outputs too.

thank you in advance for the support.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [March 4, 2018, 12:24am UTC](https://discuss.elastic.co/t/pipeline-with-persistent-queues-stops-working-after-some-hours/121841/2 "2018-03-04T00:24:24Z")

</div>

What does your config look like? How come you are using the `sleep` filter?

---

<div class="post-metadata">

**Author:** ![gghdr](https://avatars.discourse-cdn.com/v4/letter/g/e19adc/32.png) [@gghdr](https://discuss.elastic.co/u/gghdr)\
**Post date:** [March 5, 2018, 6:38pm UTC](https://discuss.elastic.co/t/pipeline-with-persistent-queues-stops-working-after-some-hours/121841/3 "2018-03-05T18:38:56Z")

</div>

My pipeline is Kafka 0.9 as Input (version 4.2.0), then in the Filter i use Sleep, Clone (2 clones), and Mutate (renaming and adding fields).  
The Outputs are Kafka (4.0.4), JMS (3.0.1) writing to IBM MQ Queue, and local file. I use the Heartbeat plugin and the Metric one for monitoring my pipeline.

The Sleep part is:  
sleep {  
time =\> "1"  
every =\> 1  
}  
then immediatly after i Clone:  
clone {  
clones =\> ["xxx","yyy"]  
}

I found out that i can replicate the problem setting the parameter **queue.max\_bytes** to a very small number (10 mb); this way Logstash stalls almost immediately. If i set this parameter to 3 gb it stalls after around 1 day.  
What i'm missing here?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 2, 2018, 6:39pm UTC](https://discuss.elastic.co/t/pipeline-with-persistent-queues-stops-working-after-some-hours/121841/4 "2018-04-02T18:39:03Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.