Mixed type lumberjack {} input


(Oneiroi) #1

Currently have a single

input {
    lumberjack {
        port => 5XXXX
        ssl_certificate => "/logstash.crt"
        ssl_key => "/logstash.key"
        codec => json
    }
}

Currently however mixed types are being sent

  • JSON logs (various)
  • syslogs
  • apache2 logs (access and error)
  • nginx logs (access and error)

In 1.5.0 it has been noted that non JSON log types carry the tag: _jsonparsefailure as a result.

As noted in https://github.com/elastic/logstash/issues/3292#issuecomment-104336293 this configuration is suboptimal and slow down is expected.

As I understand it an "optimal" configuration from the logstash perspective is ot have multiple sources in the input section e.g.

port => 1234 - syslogs (no codec => json)
port => 1235 - json logs ( codec => json)

and so on.

My question here is really, it is at all possible to have a single logstash-forwarder sending multiple log types and if so what configuration is required logstash side to avoid slow down but maintain the JSON parsing?


(system) #2