Logstash-multiple inputs one output problem

[dnastala]

Hi all,

I have a configuration misery to solve.

In my current setup I have one logstash server version 5.5.1 that takes two inputs(has to be only one logstash server).
One is from lumberjack and the other one is from brand new filebeat 6.3.0(lumberjack slowly going away but still needed)
Input config:

#################################################
input {
#collect log from lumberjacks
lumberjack {
port => "5043"
ssl_certificate => "some.crt"
ssl_key => "some.key"
}
#collect logs from filebeat
beats {
port => 5044
ssl => true
ssl_certificate_authorities => "some_ca.crt"
ssl_certificate => "some.crt"
ssl_key => "some.key"
}
}
#################################################

I need to pipe it into file output on the logstash, here's my config:

##################################################
file {
path => "/var/log/logstash-export/%{+YYYY-MM-dd}/%{host}/%{source}.log"
codec => line { format => "%{message}"}
}
##################################################

The problem is that new filebeat gives a host name in the manner:
{"name":"hostname.fqdn"}

that lives me with a bunch of directories called that way aside to the directories generated by lumberjack output.
I can fix filebeat output by modifying "path" to be:
"/var/log/logstash-export/%{+YYYY-MM-dd}/%{host[name]}/%{source}.log"

But on the other hand that is messing up output of lumberjack output since it's putting them into "host[name]" directory.

Is there any way to tag these two inputs to seperate them on the output file plugin level?

[Badger]

If I understand your question correctly, then this might solve your problem.

Alternatively, you can make the output conditional

output {
    if [data from lumberjack] {
        file {
            path => "one thing"
        }
    } else {
        file {
            path => "a different thing"
        }
    }

[dnastala]

Thanks Badger, that works good. If for example I wanted to add other inputs like tcp and syslog can I condition different outputs just for filebeat(since it would be the only one that uses json)? "[data from beats]" or "[data from filebeat]" doesn't seems to have any impact and it's treating everything as one input.

[dnastala]

got it into the perfection.

########################################################
output {
   if "name" in [host] {
       file {
            path => "/var/log/logstash-export/%{+YYYY-MM-dd}/%{host[name]}/%{source}.log"
            codec => line { format => "%{message}"}
            }
       }
   else {
       file {
            path => "/var/log/logstash-export/%{+YYYY-MM-dd}/%{host}/%{source}.log"
            codec => line { format => "%{message}"}
            }
       }
}
########################################################