Logstash crashing unexpectedly

Elastic Stack Logstash
1

Hello, logstash keeps failing and spitting out this error:

NoMethodError: undefined method `join' for "_grokparsefailure":String
  receive at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-gelf-1.0.0/lib/logstash/outputs/gelf.rb:178
  handle at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.4-java/lib/logstash/outputs/base.rb:88
  output_func at (eval):47
  outputworker at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.4-java/lib/logstash/pipeline.rb:244
  start_outputs at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.4-java/lib/logstash/pipeline.rb:166

Here's a gist of my logstash configs: https://gist.github.com/anonymous/0f05229b611b0a9d68592602d78ea32b

We're sending logs from our production app servers to a central VM in our production environment, and from there we forward them to a graylog server in our local environment.
We have logstash 1.4.2_modified on the app servers and logstash 1.5.4 on the central 'logshipper' VM.
Logstash on the central 'logshipper' VM crashes when I add a new input file to the app servers' logstash config and do some grokking and mutating. If I leave this out, logstash doesn't crash.

2

That's odd. The tags field should never be a string. That this is allowed might be a bug that's been fixed in more recent versions of Logstash. I'd try upgrading the LS 1.5.4 instance.

3

Thanks Magnus. I've upgraded the 4 servers on 1.4.2_modified to 1.5.4, and I'm getting the same error. It usually happens within a few minutes.
Maybe I should also mention that the 'logshipper' is running on centos 5.7 and the 4 app servers are running on centos 5.10.

4

While it's great that you've upgraded the ancient 1.4.2 instances, the problem is most likely introduced on the 1.5.4 which is why I suggested that you upgrade that one.

5

Oh, whoops. I thought you suggested upgrading to 1.5.4