RabbitMQ between provider and consumer


(Andrzej Rosiek) #1

Hi, I have configuration like:

nginx > logstash > rabbitmq > logstash > file

provider output config:

output {
    rabbitmq {
        id => "nginx"
        host => "rabbitmq"
        exchange => "nginx"
        exchange_type => "direct"
    }
}

consumer config:

input {
    rabbitmq {
        host => "rabbitmq"
        subscription_retry_interval_seconds => 5
    }
}

output {
    file {
        codec => rubydebug
        path => "/var/log/logstash/nginx-rabbitmq-logs-%{+YYYY-MM-dd}.log"
    }
}

provider works fine, I am able to get message in rabbitmq manager. but I am not sure why message is not passed to consumer. any ideas?

regards
Andrzej


(Magnus Bäck) #2

But you haven't configured the consumer. How would it know to start consuming from the nginx messages?


(Andrzej Rosiek) #3

what do you mean? I provided my customer config. is there something wrong with that? what kind of configuration did you mean? I am sorry I not much familiar with logstash. I just would like to figure it out.


(Magnus Bäck) #4

You've configured your rabbitmq output to send events to the nginx exchange, but you're not configuring your rabbitmq input in any way except the hostname. How would that plugin know what to listen for?


(Andrzej Rosiek) #5

Hi, thanks for the answear. as I mentioned I am using 2 containers with logstash:

  • provider
  • consumer

provider sends data to rabbitmq from nginx (via syslogs) - this part works well. I am able to get events in rabbitmq manager. I didn't put input for the provider because I think it is not imporant. am I right?

if you need provider input:

input {
  syslog {
    host => 'logstash-provider'
    port => 5140
  }
}

output {
    rabbitmq {
        id => "nginx"
        host => "rabbitmq"
        exchange => "nginx"
        exchange_type => "direct"
    }
}

and once again consumer:

input {
    rabbitmq {
        host => "rabbitmq"
        subscription_retry_interval_seconds => 5
    }
}

output {
    file {
        codec => rubydebug
        path => "/var/log/logstash/nginx-rabbitmq-logs-%{+YYYY-MM-dd}.log"
    }
}

consumer should get info from rabbitmq and put data to the file. as you may notice consumer has input and output config provided.


(Magnus Bäck) #6

consumer should get info from rabbitmq

Yes, but which RabbitMQ messages should it subscribe to? You'll have to use some combination of the exchange, queue, exchange_type, and key options. I've never used the direct exchange type so I'm not sure exactly how it works.


(system) #7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.