Lumberjack Client write error


#1

hey guys

my stack had been working fine until just recently... i have two logstash instances, one accepting courier input which uses the lumberjack output, and one accepting lumberjack input... the first logstash instance all of a sudden started throwing this error and nothing is getting through to the second logstash instance

Connecting to lumberjack server. {:addresses=>["elk-stage-esclient-01.ins.us-west-2.i.identityaws.net"], :port=>4546, :ssl_certificate=>"/etc/pki/ca-trust/source/anchors/logstash-stage-esclient.crt", :window_size=>nil, :level=>:info}
Sending events to lumberjack {:size=>1, :level=>:info}
Client write error, trying connect {:e=>#<RuntimeError: Whoa we shouldn't get this frame: >, :backtrace=>["/opt/logstash/vendor/bundle/jruby/1.9/gems/jls-lumberjack-0.0.24/lib/lumberjack/client.rb:132:in ack'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/jls-lumberjack-0.0.24/lib/lumberjack/client.rb:120:inwrite_sync'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/jls-lumberjack-0.0.24/lib/lumberjack/client.rb:41:in write'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-lumberjack-1.0.2/lib/logstash/outputs/lumberjack.rb:71:inflush'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.21/lib/stud/buffer.rb:219:in buffer_flush'", "org/jruby/RubyHash.java:1341:ineach'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.21/lib/stud/buffer.rb:216:in buffer_flush'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.21/lib/stud/buffer.rb:159:inbuffer_receive'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-lumberjack-1.0.2/lib/logstash/outputs/lumberjack.rb:54:in register'", "org/jruby/RubyProc.java:271:incall'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-codec-json_lines-1.0.1/lib/logstash/codecs/json_lines.rb:45:in encode'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-lumberjack-1.0.2/lib/logstash/outputs/lumberjack.rb:65:inreceive'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.4-java/lib/logstash/outputs/base.rb:88:in handle'", "(eval):56:inoutput_func'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.4-java/lib/logstash/pipeline.rb:244:in outputworker'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.4-java/lib/logstash/pipeline.rb:166:instart_outputs'"], :level=>:error}
Connecting to lumberjack server. {:addresses=>["elk-stage-esclient-01.ins.us-west-2.i.identityaws.net"], :port=>4546, :ssl_certificate=>"/etc/pki/ca-trust/source/anchors/logstash-stage-esclient.crt", :window_size=>nil, :level=>:info}


(system) #2