Hello,
I have the following setup:
[Nginx Port 80]
Forwards to
[Logstash Port 5044]
Directly connects and speaks with
[Elasticsearch Port 9200]
This works with Metricbeat fine, all requests and conversations happen.
Here is my nginx setup:
user root;
worker_processes auto;
load_module /usr/lib/nginx/modules/ngx_stream_module.so;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
stream {
log_format basic '$time_iso8601 $remote_addr '
'$protocol $status $bytes_sent $bytes_received '
'$session_time $upstream_addr '
'"$upstream_bytes_sent" "$upstream_bytes_received" "$upstream_connect_time"';
# Enable access_log statements for debugging
access_log /var/log/nginx/stream.log basic;
upstream syslog_servers {
server logstash:514;
}
server {
listen 514;
listen 514 udp;
proxy_responses 0;
proxy_pass syslog_servers;
proxy_buffer_size 4096k;
# access_log /var/log/nginx/stream.log basic;
}
upstream logstash_beats_servers {
server logstash:5044;
}
server {
listen 80;
proxy_responses 1;
proxy_connect_timeout 30s;
proxy_pass logstash_beats_servers;
proxy_buffer_size 4096k;
access_log /var/log/nginx/winlogbeat.log basic;
}
}
daemon off;
Here is my beats configuration:
input {
beats {
port => 5044
type => beats
}
}
output {
if [type] == "beats" {
elasticsearch {
hosts => "elasticsearch:9200"
manage_template => false
index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}
}
I have captured the error in Nginx:
2018/05/15 14:23:50 [error] 1794#1794: *527214 recv() failed (104: Connection reset by peer) while proxying connection, client: 192.168.1.10, server: 0.0.0.0:80, upstream: "172.18.0.10:5044", bytes from/to client:3876/18, bytes from/to upstream:18/3876
2018/05/15 14:28:01 [error] 1795#1795: *528980 recv() failed (104: Connection reset by peer) while proxying connection, client: 192.168.1.10, server: 0.0.0.0:80, upstream: "172.18.0.11:5044", bytes from/to client:1669/6, bytes from/to upstream:6/1669
2018/05/15 14:29:36 [error] 1795#1795: *529380 recv() failed (104: Connection reset by peer) while proxying connection, client: 192.168.1.10, server: 0.0.0.0:80, upstream: "172.18.0.12:5044", bytes from/to client:3196/12, bytes from/to upstream:12/3196
I have also captured the problem in Winlogbeat log:
2018-05-15T16:05:56.893+0100 ERROR logstash/async.go:235 Failed to publish events caused by: EOF
2018-05-15T16:05:56.895+0100 ERROR logstash/async.go:235 Failed to publish events caused by: client is not connected
2018-05-15T16:05:57.896+0100 ERROR pipeline/output.go:92 Failed to publish events: client is not connected
I am unsure what the difference in the conversation is between Metricbeat -> Logstash and Winlogbeat -> Logstash, I was of the understanding they both used the lumberjack protocol on 5044