Hi everyone, I need some help.
I have configured my ES stack inside a docker. I need to show two types of logs (syslog, log), in an Elastic index. I have created two configuration files
- logstash.cong
- logstash-syslog.conf
I also created two pipelines in logstash to manage the two files. Despite this Elasticsearch always shows only the logs of one of the two and never both together.
Configuration files below
logstash.cong
input {
beats {
port => 5044
}
tcp {
port => 5000
}
}
filter {
mutate { replace => { "[host]" => "%{[host][name]}" } }
}
output {
elasticsearch {
hosts => "https://elasticsearch:9200"
user => "user"
password => "password"
ecs_compatibility => disabled
cacert => "config/elasticsearch-ca.pem"
#index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
index => "my-index"
}
}
logstash-syslog.conf
input {
tcp {
port => 5044
type => syslog
}
udp {
port => 5044
type => syslog
}
}
filter {
if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
}
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}
output {
elasticsearch {
hosts => "https://elasticsearch:9200"
user => "user"
password => "password"
ecs_compatibility => disabled
cacert => "config/elasticsearch-ca.pem"
#index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
index => "my-index"
}
}
pipelines.yml
- pipeline.id: pipeline_1
path.config: "/usr/share/logstash/pipeline/logstash.conf"
pipeline.workers: 3
- pipeline.id: pipeline_2
path.config: "/usr/share/logstash/pipeline/logstash-syslog.conf"
queue.type: persisted
How can i report the two types of logs in the elastic index?
Thanks in advance