Logstash Multiple files config


(Bruno De Oliveira Bastos) #1

Hi, i try to make multiple files for every service. I have two files ( squid.conf and dansguardian.conf ). When i execute logstash with only one config file, i have 63 fiels in Squid and 33 in Dansguardian. But if if put all config files in both i have only 33 fiels ( squid and dansguardian ). I dont know why. If someone can help me, this is my files

Squid.conf

input {
file {
path => ["/var/log/squid/access.log"]
type => "squid"
}
}
filter {
grok {
patterns_dir => ["/etc/logstash/conf.d/patterns"]
match => [ "message", "%{DATA:timestamp} %{NUMBER:response_time:int} %{IP:src} %{WORD:cache_result}/%{NUMBER:status_code:int} %{NUMBER:bytes:int} %{WORD:method} %{NOTSPACE:url} %{USER:username} %{WORD:squid}/(?:-|%{IP:dst_ip}) %{NOTSPACE:content_type}" ]
add_tag => ["squid"]
}
geoip {
source => "dst_ip"
}
}
output
{
elasticsearch {
action => "index"
user => "elastic"
password => "system"
hosts => ["127.0.0.1"]
document_type => "squid-%{+YYYY.MM.dd}"
index => "squid"
}
}

Dansguardian.conf

input {
file {
path => ["/var/log/dansguardian/access.log"]
type => "dansguardian"
}
}
filter {
grok {
patterns_dir => ["/etc/logstash/conf.d/patterns"]
match => [ "message", "%{DATA:timestamp} %{NUMBER:response_time:int} %{IP:src} %{WORD:cache_result}/%{NUMBER:status_code:int} %{NUMBER:bytes:int} %{WORD:method} %{NOTSPACE:url} %{USER:username}" ]
add_tag => ["dansguardian"]
}
}
output
{
elasticsearch {
action => "index"
user => "elastic"
password => "system"
hosts => ["127.0.0.1"]
document_type => "dansguardian-%{+YYYY.MM.dd}"
index => "dansguardian"
}
}


(Mark Walkom) #2

LS merges multiple files, you should look at https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html and setup some conditionals.


(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.