Logstash RabbitMQ input plugin configuration

Elastic Stack Logstash
1

Hello,

My goal is:
1: Configure Logstash (version: 6.2.1) to pull events from RabbitMQ (version: 3.6.1) ( as input)
2: Depending upon the content of the event (message), store them in different files (as output)

Right now, I'm struggling to achieve my first goal. Here's my logstash config:
input {
rabbitmq {
host => "localhost"
durable => true
exchange => "RSLOG"
queue => "RSLOG"
}
}

output {
stdout { }
}

After starting logstash, the log, along with some startup messages prints this:
[2019-01-29T09:41:47,284][ERROR][logstash.inputs.rabbitmq ] RabbitMQ connection error, will retry. {:error_message=>"Connection to localhost:5672 failed: timeout", :exception=>"MarchHare::ConnectionRefused"}

I tried adding port to the configuration. But no mater what port number I add, logstash prints the same message. There no service on my local box which is listening on port 5672.
What is this port? Is this the port where rabbit listens on for admin console (15672)?

Documentation says:

This plugin uses the March Hare library for interacting with the RabbitMQ server.

Unfortunately I didn't find anywhere how to install and configure it.

After I get my goal 1 working, is it possible to achieve my goal 2?

Any help is greatly appreciated!!

Thanks!
Mangesh

2

No, 15672 is the admin interface. 5672 is the port used to talk AQMP. Have you configured a listener in your RabbitMQ configuration?

3

Hello @Badger

That was the missing link. I enabled the tcp_listener at 5672. This time, logstash could not authenticate. Here's the message:

[2019-01-29T11:41:50,274][ERROR][logstash.inputs.rabbitmq ] RabbitMQ connection error, will retry. {:error_message=>"Authentication with RabbitMQ failed or RabbitMQ version used does not support AMQP 0-9-1. Username: guest, vhost: /, password length: 5. Please check your configuration.", :exception=>"MarchHare::AuthenticationFailureError"}

Thanks!

4

Hello @Badger

That worked! Now, logstash is listening on a queue for any event and the moment it receives it, it dumps it on the console.

The event (message) will be a text and will have a component name from where it got sent. My other goal is to parse it and log it in different files.

Do you think it's doable using logstash?

Thanks in advance!

5

That should be doable using logstash. Look at the dissect and grok filters for parsing. You can use a file output and have the filename depend on the contents of the message. For example, this configuration will put one line in /tmp/foo2 and one in /tmp/foo3.

input { generator { count => 1 message => '{ "target": "foo2" }' } }
input { generator { count => 1 message => '{ "target": "foo3" }' } }
filter { json { source => "message" } }
output { file { path => "/tmp/%{target}" } }
6

Hi @Badger,

That's great information. It educated me and saved a lot of my time already!

A Very Big Thank You!!

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.