scenario:
rabbitmq have different queues more than three. I want to make logstash configuration to get data from the rabbitmq queue and index this data to Elasticsearch. In order to that I have created logstash conf files for EVERY queue SEPARATELY with below configuration.
input {
rabbitmq {
id => "rabbitmyq_id0"
# connect to rabbit
host => "rabbitmq_hostname"
port => 5672
vhost => "/"
user => "user"
password => "pass"
queue => "WebApiLog"
ack => false
durable => true
exchange => "logstash_WebApiLog"
exchange_type => "fanout"
key => "WebApiLog"
arguments => {
"x-queue-type" => "quorum"
}
}
}
output {
elasticsearch {
hosts => [ "https://elastichostname:9200" ]
index => "webapi-logs-%{+YYYY.MM.dd}"
}
}
When I run the logstash. All of the rabbit exchange get in place and logstash start to collect data from rabbitmq and index this data to elasticsearch.
The problem:
Although I created separate config for every queues separately. When I check the created indices in elastic , I see that every indices has data for ALL queues, not just data of queue it created. I want to do every indices has data that belong to queue it is created for.
Can someone help me, how I can do that logstash configuration.
