RabbitMQ to LogStash / Elasticsearch

Elastic Stack Logstash
1

I m very new to ElasticSearch and LogStash. and I want to move messages out from RabbitMQ for ElasticSearch and further Kibana to visualize aggregations.

Application Messages >> RabbitMQ >> LogStash >> Index in ElasticSearch >> Visualize on Kibana.

On windows server, I installed ElasticSearch and LogStash 2.1.1.

Now I do understand that there is a rabbitmq plugin
https://www.elastic.co/guide/en/logstash/current/plugins-inputs-rabbitmq.html

Logstash error
I tried to install and get this error on LogStash.
C:\logstash-2.1.1\bin>plugin install rabbitmq --verbose
rabbitmq is not a Logstash plugin
ERROR: Installation aborted, verification failed for rabbitmq

Elasticsearch error
I then tried to install on Elasticsearch
C:\elasticsearch-2.1.1\bin>plugin install rabbitmq --verbose
-> Installing rabbitmq...
Trying https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugi
n/rabbitmq/2.1.1/rabbitmq-2.1.1.zip ...
Failed: FileNotFoundException[https://download.elastic.co/elasticsearch/release/
org/elasticsearch/plugin/rabbitmq/2.1.1/rabbitmq-2.1.1.zip]; nested: FileNotFoun
dException[https://download.elastic.co/elasticsearch/release/org/elasticsearch/p
lugin/rabbitmq/2.1.1/rabbitmq-2.1.1.zip];
ERROR: failed to download out of all possible locations..., use --verbose to get

  • detailed information*
2

Hi. Thought I'd try this to see if it worked for me. It didn't. Same
error :frowning:

Some googling led me to think that the rabbitmq plugin needs to be
downloaded - but how?

Eventually found this: (modify the path to your ruby install under you
logstash folder if needed)

C:\Elastic\logstash\vendor\jruby\bin>gem fetch logstash-input-rabbitmq
io/console not supported; tty will not be manipulated
Fetching: logstash-input-rabbitmq-3.1.3.gem (100%)
Downloaded logstash-input-rabbitmq-3.1.3

Then this:

C:\Elastic\logstash\bin>plugin install logstash-input-rabbitmq
io/console not supported; tty will not be manipulated
Validating logstash-input-rabbitmq
Installing logstash-input-rabbitmq
Installation successful

Haven't tried the plugin myself yet, but hope this helps.

3

Thank you, so much that did get it installed.
One more question from the newbie; where do I put the config information of RabbitMQ host/port/login etc. in logstash?

Thank you once again.

4

One more question from the newbie; where do I put the config information of RabbitMQ host/port/login etc. in logstash?

Put a rabbitmq block in any input block in any Logstash file.

input {
  rabbitmq {
    ...
  }
}
5

Hi Sir,

I am new to DevOps field.
Even I am having the same requirements as yours.
That is I need to send the RabbitMq logs to Logstash and Visualize in Kibana.

But I Dont know how to move forward.
I have installed Kibana 5, Logstash 5.X, Elastc Search and Filebeat.

I am confused how to move forward.

Please help to move forward.

Thanks in Advance,
Roopa K