Hi All,
I am having trouble with my filebeat/logstash. connectivity seems to be inconsistenst. When i try it in debug mode it works fine but when i use it with systemctl start it fails . Below are my filebeat and logstash config details and error output details
Filebeat:
filebeat.inputs:
Each - is an input. Most options can be set at the input level, so
you can use different inputs for various configurations.
Below are the input specific configurations.
type: log
Change to true to enable this input configuration.
enabled: true
Paths that should be crawled and fetched. Glob based paths.
paths:
"/home/clouduser/apache-tomcat-9.0.36/logs/catalina.*.log"
ignore_older: 24h
#- c:\programdata\elasticsearch\logs*
============================== Filebeat modules ==============================
filebeat.config.modules:
enabled: true
path: ${path.config}/modules.d/*.yml
------------------------------ Logstash Output -------------------------------
output.logstash:
The Logstash hosts
hosts: ["127.0.0.1:5044"]
bulk_max_size: 1024
and my Logstash beats.conf file
input {
beats {
client_inactivity_timeout => 1200
port => 5044
ssl => false
ssl_certificate => "/etc/pki/tls/certs/logstash.crt"
ssl_key => "/etc/pki/tls/private/logstash.key"
}
}
output {
file {
path => "/mnt/xxxxxxxxxxxxx/Logs/Dev_Internal/VMSailPointTaskDEV0"
path => "/mnt/xxxxxxxxxxx/Logs/Dev_Internal/Ganeshtest1TaskDev1/%{[host][name]}-catalina_logs-%{+YYYY-MM-dd}.log"
codec => line { format => "%{message}" }
}
and logs from filebeat
2020-09-30T23:15:50.798-0400 DEBUG [transport] transport/client.go:205 handle error: write tcp 127.0.0.1:54730->127.0.0.1:5044: write: connection reset by peer
2020-09-30T23:15:50.798-0400 DEBUG [transport] transport/client.go:118 closing
2020-09-30T23:15:50.798-0400 DEBUG [logstash] logstash/async.go:172 73 events out of 73 events sent to logstash host 127.0.0.1:5044. Continue sending
2020-09-30T23:15:50.798-0400 DEBUG [logstash] logstash/async.go:128 close connection
2020-09-30T23:15:50.798-0400 ERROR [logstash] logstash/async.go:280 Failed to publish events caused by: write tcp 127.0.0.1:54730->127.0.0.1:5044: write: connection reset by peer
2020-09-30T23:15:50.798-0400 DEBUG [logstash] logstash/async.go:128 close connection
2020-09-30T23:15:50.798-0400 INFO [publisher] pipeline/retry.go:219 retryer: send unwait signal to consumer
2020-09-30T23:15:50.798-0400 INFO [publisher] pipeline/retry.go:223 done
2020-09-30T23:15:50.798-0400 DEBUG [harvester] log/log.go:107 End of file reached: /home/clouduser/apache-tomcat-9.0.36/logs/catalina.2020-09-30.log; Backoff now.
2020-09-30T23:15:52.532-0400 ERROR [publisher_pipeline_output] pipeline/output.go:180 failed to publish events: write tcp 127.0.0.1:54730->127.0.0.1:5044: write: connection reset by peer
2020-09-30T23:15:52.532-0400 INFO [publisher_pipeline_output] pipeline/output.go:143 Connecting to backoff(async(tcp://127.0.0.1:5044))
2020-09-30T23:15:52.532-0400 DEBUG [logstash] logstash/async.go:120 connect
2020-09-30T23:15:52.533-0400 INFO [publisher] pipeline/retry.go:219 retryer: send unwait signal to consumer
2020-09-30T23:15:52.533-0400 INFO [publisher] pipeline/retry.go:223 done
2020-09-30T23:15:52.798-0400 DEBUG [harvester] log/log.go:107 End of file reached: /home/clouduser/apache-tomcat-9.0.36/logs/catalina.2020-09-30.log; Backoff now.
2020-09-30T23:15:54.767-0400 DEBUG [input] input/input.go:139 Run input
2020-09-30T23:15:54.767-0400 DEBUG [input] log/input.go:205 Start next scan
2020-09-30T23:15:54.767-0400 DEBUG [input] log/input.go:439 Check file for harvesting: /home/clouduser/apache-tomcat-9.0.36/logs/catalina.2020-09-30.log
2020-09-30T23:15:54.767-0400 DEBUG [input] log/input.go:530 Update existing file for harvesting: /home/clouduser/apache-tomcat-9.0.36/logs/catalina.2020-09-30.log, offset: 169191
2020-09-30T23:15:54.767-0400 DEBUG [input] log/input.go:582 Harvester for file is still running: /home/clouduser/apache-tomcat-9.0.36/logs/catalina.2020-09-30.log
2020-09-30T23:15:54.767-0400 DEBUG [input] log/input.go:226 input states cleaned up. Before: 1, After: 1, Pending: 0
Can some one help me , why I am facing network/connectivity issues from logstash to filebeatPlease don't create multiple topics on the same question - Filebeat is not sending logs to Logstash, logstash is localhost