Multiple output in logstash config, pls help!

I managed to put multiple inputs/filters/outputs in one logstash conf, with no error or warning while running elasticsearch and logstash, but there is only one index in elasticsearch, pls help:

my con is as below:

input {
beats {
port => "5044"
type => "applog"
}
udp {
port => "514"
type => "syslog"
}
}
filter {
if [type] == "applog" {
grok {
match => [ "message", "%{COMBINEDAPACHELOG}" ]
}
date {
match => [ "logdate", "dd/MMM/yyyy:HH:mm:ss Z" ]
}
mutate {
remove_field => [ "logdate" ]
}
}
if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:[%{POSINT:syslog_pid}])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
}
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}
output {
if [type] == "applog" {
elasticsearch {
hosts => [ "192.168.11.2:9200" ]
index => "applog.logstash-%{+YYYY.MM.dd}"
}
}
if [type] == "syslog" {
elasticsearch {
hosts => [ "192.168.11.2:9200" ]
index => "syslog.logstash-%{+YYYY.MM.dd}"
}
}
}

as curl to elasticsearch, I got this:

health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open .kibana fQuowSJwRMO8uQ17yzmqlw 1 1 2 0 10.1kb 10.1kb
yellow open syslog.logstash-2016.11.19 PEsD8uhNSVSLj1U3hxkaJQ 5 1 1501 0 878.3kb 878.3kb

what's wrong and what should I do?

Is Logstash receiving anything via the beats input? What do the logs on the sending side look like (Filebeat or whatever it is you're using)?

the sending side is : filebeat for sending nginx(access log), and winlogbeat for sending windows eventlog.

Yes, and what do the logs of those applications say?

No, I'm asking for the log of Filebeat itself. Is there any sign in its log that it's either successful or not successful in sending data to Logstash?

Wow!, just check the filebeat log, it showed:

2016-11-21T20:59:18+08:00 INFO No non-zero metrics in the last 30s
2016-11-21T20:59:48+08:00 INFO No non-zero metrics in the last 30s
2016-11-21T21:00:18+08:00 INFO No non-zero metrics in the last 30s
2016-11-21T21:00:48+08:00 INFO No non-zero metrics in the last 30s
2016-11-21T21:01:18+08:00 INFO No non-zero metrics in the last 30s
2016-11-21T21:01:48+08:00 INFO No non-zero metrics in the last 30s
2016-11-21T21:02:18+08:00 INFO No non-zero metrics in the last 30s
2016-11-21T21:02:48+08:00 INFO No non-zero metrics in the last 30s
2016-11-21T21:03:18+08:00 INFO No non-zero metrics in the last 30s

Finally got this solved by using filebeat to elasticsearch directly, after restarting filebeat, everything is fine.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.