RabbitMQ output + Compress spooler: "wrong number of arguments"

Elastic Stack Logstash
1

I'm trying to use the compress spooler codec on a previously-working RMQ output, but encoding is failing with Error encoding event {:exception=>#<ArgumentError: wrong number of arguments (1 for 2)> ... .

I've stripped my config down to basic input/output and can reproduce it with the following:

input {
generator {
count => 10
message => "test"
}
}

output {

rabbitmq {
codec => compress_spooler {
spool_size => 10
}
exchange => "from-ingress"
exchange_type => "direct"
host => "dev-queue1"
key => "filter-routing-key"
ssl => "true"
port => "5671"
user => "logstash_user"
password => "[not_telling]"
durable => "true"
persistent => "true"
}
}

It seems to be an interaction between rabbitmq and compress_spooler, as other outputs I've tried (file, stdout, tcp) don't throw this error.

Can anyone else reproduce this and maybe shed some light on the cause?

2

What version of LS and the plugin are you using?
What OS (and version)?
What Java version and release?

3

Hey @warkolm

Not sure why I didn't include versions :confused:

Ubuntu 14.04.3
LS 2.0.0-1
logstash-output-rabbitmq 3.0.5
logstash-codec-compress_spooler 2.0.2
Openjdk 7u79-2.5.6-0ubuntu1.14.04.1

4

*bump*

@warkolm - any ideas?

5

I think this would be worth raising as an issue here - https://github.com/logstash-plugins/logstash-output-rabbitmq/issues

6

@ceekay - any updates? are u still having same issue? I am reproducing the same behavior when logstash is communicating to Rabbitmq using compress_spooler...
it works without compress_spooler but I don't want to submit uncompressed packages that contain only one line to Rabbitmq... did you find any alternative?

7

@oriol we ended up dropping compress_spooler and writing our own codec with similar functionality. This also needs a slightly tweaked RabbitMQ plugin to ensure there's no message loss on shutdown, and we're still waiting on PR acceptance for that so we can't really release what we have until that happens.

I'll update this thread when there's some movement there.

8

still no updates on this issue? I'm having the same problem with RabbitMQ 3.6.6, Logstash 2.4.1 and logstash-codec-compress_spooler 2.0.4