2 types of input

Elastic Stack Logstash
1

Hi,

I'm using a logstash-utm.conf file to process syslog data from a firewall, this is working fine.
I also want to send data from winlogbeat on a windows server to logstash, is it best practice to add new input and output to the existing logstash-utm.conf?

This is my current conf file:

input {
tcp {
port => 5140
}
udp {
port => 5140
}
}

filter {
grok {
match => { "message" => "<%{POSINT:syslog_pri}>%{DATA:syslog_timestamp} %{SYSLOGHOST:syslog_host} %{DATA:syslog_program}[%{NUMBER:syslog_pid}]: %{GREEDYDATA:syslog_message}" }
}
date {
match => [ "syslog_timestamp", "yyyy:MM:dd-HH:mm:ss" ]
}
kv {
source => "syslog_message"
}
mutate {
replace => [ "type", "%{syslog_program}" ]
remove_field => [ "syslog_message", "syslog_timestamp" ]
}

if [type] == "httpproxy" {
grok { match => { "url" => "(?https?)://%{IPORHOST:url_domain}/" } }

}
geoip {
source => "dstip"
}
geoip {
source => "srcip"
}

} # end of filter

output {

elasticsearch {
hosts => ["10.255.254.27:9200"]
index => "logstash-utm-%{+YYYY.MM.dd}"

}

stdout { codec => rubydebug }

}

2

You can have many instances of the same input in one config but obviously they can't "step on each others toes" i.e. have the same listening port, reading the same files or fetching the same records.

You should consider using the type setting. It will add a field called type that you can use in conditionals to apply some logic to one source and not the other.

Also you can use multiple pipelines in one instance to separate the concerns. Two input + filter pipelines can feed into a common output pipeline. Here is the link to the multi-pipeline docs

3

Something like this?

input {
tcp {
port => 5140
}
udp {
port => 5140
beats {
type => "winlogbeat"
port => 5044
}

filter {
grok {
match => { "message" => "<%{POSINT:syslog_pri}>%{DATA:syslog_timestamp} %{SYSLOGHOST:syslog_host} %{DATA:syslog_program}[%{NUMBER:syslog_pid}]: %{GREEDYDATA:syslog_message}" }
}
date {
match => [ "syslog_timestamp", "yyyy:MM:dd-HH:mm:ss" ]
}
kv {
source => "syslog_message"
}
mutate {
replace => [ "type", "%{syslog_program}" ]
remove_field => [ "syslog_message", "syslog_timestamp" ]
}

if [type] == "httpproxy" {
grok { match => { "url" => "(?https?)://%{IPORHOST:url_domain}/" } }

}
geoip {
source => "dstip"
}
geoip {
source => "srcip"
}

} # end of filter

output {
if [type] == "winlogbeat" {
elasticsearch { index => "logstash-winlogbeat-%{YYYY.mm.dd}"
} else {
elasticsearch {
hosts => ["10.255.254.27:9200"]
index => "logstash-utm-%{+YYYY.MM.dd}"

}

stdout { codec => rubydebug }

}

4

Sure, that is OK (except there is a typo, no closing } for udp) but I mean exactly the same input plugin, e.g.:

input {
  tcp {
    port => 5140
    type => "app-A"
  }
  tcp {
    port => 5141
    type => "app-B"
  }
}
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.