# Logstash stop reading from RabbitMQ

**URL:** <https://discuss.elastic.co/t/logstash-stop-reading-from-rabbitmq/34750>\
**Category:** Logstash\
**Created:** [November 17, 2015, 12:59am UTC](https://discuss.elastic.co/t/logstash-stop-reading-from-rabbitmq/34750 "2015-11-17T00:59:59Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![srikanth.guduru](https://avatars.discourse-cdn.com/v4/letter/s/ecc23a/32.png) [@srikanth.guduru](https://discuss.elastic.co/u/srikanth.guduru)\
**Post date:** [November 17, 2015, 12:59am UTC](https://discuss.elastic.co/t/logstash-stop-reading-from-rabbitmq/34750/1 "2015-11-17T00:59:59Z")

</div>

After sometime logstash stop reading data from RabbitMQ.

Gives out no error, just halts. Messages stack-up in unacknowledged mode in RabbitMQ

Messages rates can go upto 40-50 messages per second.

My config for losgtash is:

input {  
rabbitmq {  
host =\> "xxxxxxx"  
user =\> "xxxxxxx"  
password =\> "xxxxxxx"  
port =\> 5672  
exchange =\> "amq.direct"  
vhost =\> "xxxxx"  
queue =\> "xxxxx"  
durable =\> true  
auto\_delete =\> false  
type =\> "xxxxxx"  
threads =\> 2  
}  
}

filter {  
if [type] == "xxxxxxx" {  
grok {  
match =\> ["message", "%{TIMESTAMP\_ISO8601:time\_stamp} %{DATA:deviceId} %{DATA:deviceName} - \s\*%{GREEDYDATA:message}"]  
overwrite =\> ["message"]  
}

```
date {
  match => ["time_stamp", "ISO8601", "YYYY-MM-dd HH:mm:ssZ"]
  remove_field => ["time_stamp"]
}

if("<" in [message] and ">" in [message]) {
  drop{} 
}

```

}  
}

output {  
if [type] == "xxxxxxx" {  
elasticsearch {  
hosts =\> ["xxxxxxxxx:9200"]  
index =\> "xxxxxxx-%{+YYYY.MM.dd}"  
}

```
stdout { }

```

}  
}

 ![](https://us1.discourse-cdn.com/elastic/original/2X/6/6442af0081040667e8a008ccc05aedb62a34397e.png)

Any help will be greatly appreciated

even I added 16 worker threads but still problem is same.

If I restart logstash it will work for few hours (I will loose some messages also) but again problem remains same.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [November 17, 2015, 6:56am UTC](https://discuss.elastic.co/t/logstash-stop-reading-from-rabbitmq/34750/2 "2015-11-17T06:56:47Z")

</div>

Remove some complexity to debug this.

- Drop `threads` so 1.
- Drop all filters.
- Replace the elasticsearch output with e.g. a file output that just dumps the messages.

You can do this in a separate Logstash instance. Does it help? If yes, reintroduce feature by feature until it breaks again.

---

<div class="post-metadata">

**Author:** ![srikanth.guduru](https://avatars.discourse-cdn.com/v4/letter/s/ecc23a/32.png) [@srikanth.guduru](https://discuss.elastic.co/u/srikanth.guduru)\
**Post date:** [November 18, 2015, 7:58pm UTC](https://discuss.elastic.co/t/logstash-stop-reading-from-rabbitmq/34750/3 "2015-11-18T19:58:33Z")

</div>

Hey Magnus,

Thank you very much responding so quick.

and thank you for the advice.

Ya I followed the steps you mentioned.. and Voila!! I found that after removing elasticsearch from output the halting problem solved.

But now the question is how can I resolve this. I need to use elasticsearch as database.

Any advice ??

Regards  
Srikanth

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [November 19, 2015, 6:47am UTC](https://discuss.elastic.co/t/logstash-stop-reading-from-rabbitmq/34750/4 "2015-11-19T06:47:32Z")

</div>

Have you looked for anything interesting in the Logstash log? What if you crank up the log level with `--verbose` or possible even `--debug`? What about the Elasticsearch log?

---

<div class="post-metadata">

**Author:** ![srikanth.guduru](https://avatars.discourse-cdn.com/v4/letter/s/ecc23a/32.png) [@srikanth.guduru](https://discuss.elastic.co/u/srikanth.guduru)\
**Post date:** [December 1, 2015, 7:24pm UTC](https://discuss.elastic.co/t/logstash-stop-reading-from-rabbitmq/34750/5 "2015-12-01T19:24:14Z")

</div>

Hi Magnus, Happy Thanks Giving...

Replying you after long time.

Ya I tried to find out the problem but didnt find any appropriate answer. After some reading I realized that its something related to elasticsearch bulk API.

Basically I deployed all logstash instances in one VM and elasticsearch instance in other VM. Now I deployed the elasticsearch and logstash(s) together in one VM. Now its working fine.. Until now it didnt halt..

But still looking for an optimal architecture/hierarchy.

Thank you..

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:20am UTC](https://discuss.elastic.co/t/logstash-stop-reading-from-rabbitmq/34750/6 "2017-07-06T05:20:31Z")

</div>