Host field not present on logstash event

Elastic Stack Logstash
1

Greetings.

I am shipping logs from two servers in disparate data centers and utilizing the lumberjack plugin as a means of communication between the two. The communication is working fine (no ssl issues) but when the logstash event is sent from server A to server B, the host field is not on the logstash event.

Details:
java version "1.8.0_65"
logstash 2.1.0
DISTRIB_DESCRIPTION="Ubuntu 14.04.1 LTS"

server A config:

input {
  file {
    path => '/var/log/auth.log'
  }
}
output {
  lumberjack {
    hosts => ["1.1.1.1"]
    port => 6782
    ssl_certificate => "/etc/ssl/certs/redacted.crt"
  }
}

server B config:

input {
  lumberjack {
    port => 6782
    ssl_certificate => "/etc/ssl/certs/redacted.crt"
    ssl_key => "/etc/ssl/private/redacted.key"
  }
}
output {
  file {
    path => './test-fun.txt'
  }
}

Sample event that was generated on server A, and sent via lumberjack to server B

{"message":"2015-12-24T04:13:46.224Z redacted.net Dec 24 04:13:45 redacted sshd[4743]: Received disconnect from 2.2.2.2: 11: Bye! [preauth]","@version":"1","@timestamp":"2015-12-24T04:13:46.297Z"}

As you can see from the message, there is no host field. As a test, I created an output file on server A and looked at the event before it was sent to server B and sure enough, the host field is there. It's as if lumberjack on the receiving side (lumberjack input plugin) is deleting the host field.

Anyone have any thoughts? Should I switch to the "beats" plugin for the input on server B?

Thanks,

Matthew Green

2

Use the json codec in both the output and input lumberjack plugin. Right now you're using the default plain codec so Logstash is serializing the whole event to a string equivalent to "%{@timestamp} %{message}" so all other fields are lost.