Pipeline-to-pipeline with single input and output

Elastic Stack Logstash
1

Hi , I'm trying to configure logstash with pipelines configuration

logstash pipelines:

input --> conf files with different log types (system,security, auditing ...) --> output

but it doesn't work(without errors ) , what I'm doing wrong

beats_input.conf(input):

input {
beats {
port => 5044
client_inactivity_timeout => 600
}
}

output {
if [fileset][module] == "system" {
if [fileset][name] == "auth" {
pipeline { send_to => system_auth }
}
if [fileset][name] == "syslog" {
pipeline {send_to => system_syslog }
}
}
}

system.conf:

input {

pipeline {address => system_syslog}

}

filter {

  grok {
     add_field => { "[@metadata][index]" => "system-syslog" }

    match => { "message" => ["%{SYSLOGTIMESTAMP:[system][syslog][timestamp]} %{SYSLOGHOST:[system][syslog][hostname]} %{DATA:[system][syslog][program]}(?:\[%{POSINT:[system][syslog][pid]}\])?: %{GREEDYMULTILINE:[system][syslog][message]}"] }
    pattern_definitions => { "GREEDYMULTILINE" => "(.|\n)*" }
    remove_field => "message"
  }
  date {
    match => [ "[system][syslog][timestamp]", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
  }
}

output {
pipeline{send_to=>elasticsearch}

}

elasticsearch(output):

input {
pipeline{address => elasticsearch}

}

output {
elasticsearch {
hosts => [ "il-infra-es1", "il-infra-es2", "il-infra-es3" ]
manage_template => false
index => "%{[@metadata][index]}-%{+YYYY.MM.dd}"
}
}

pipelines.yml:

  • pipeline.id: beats_input
    path.config: "/etc/logstash/conf.d/beats_input.conf"
  • pipeline.id: system-auth
    path.config: "/etc/logstash/conf.d/system_auth.conf"
  • pipeline.id: system-syslog
    path.config: "/etc/logstash/conf.d/system_syslog.conf"
  • pipeline.id: elasticsearch
    path.config: "/etc/logstash/conf.d/elasticsearch.conf"
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.