Hi,
Please I am seeing logs of "No connection could be made because the target machine actively refused it.". The logstash pipeline is running. Any guidance is much appreciated. Thanks.
My Logstash pipeline config is as below. I am sending Windows / Nxlogs to port 3521.
<
NXlog for Windows Events
tcp {
port => 3521
ssl_enable => true
ssl_cert => "/usr/share/logstash/config/logstash-forwarder.crt"
ssl_key => "/usr/share/logstash/config/logstash-forwarder.key"
ssl_verify => false
codec => json_lines { charset => CP1252 }
tags => ["eventlog"]
}
/>
When I do a curl from the client machine running Nxlog to the Logstash server this is the output below.
<
C:\curl>curl -v --cacert logstash-forwarder.crt https://proteus.xxxx.com:3521
- Trying 192.168.0.82...
- TCP_NODELAY set
- Connected to proteus.xxxx.com (192.168.0.82) port 3521 (#0)
- ALPN, offering h2
- ALPN, offering http/1.1
- successfully set certificate verify locations:
- CAfile: logstash-forwarder.crt
CApath: none - TLSv1.3 (OUT), TLS handshake, Client hello (1):
- TLSv1.3 (IN), TLS handshake, Server hello (2):
- TLSv1.2 (IN), TLS handshake, Certificate (11):
- TLSv1.2 (IN), TLS handshake, Server key exchange (12):
- TLSv1.2 (IN), TLS handshake, Server finished (14):
- TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
- TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
- TLSv1.2 (OUT), TLS handshake, Finished (20):
- TLSv1.2 (IN), TLS handshake, Finished (20):
- SSL connection using TLSv1.2 / ECDHE-RSA-AES256-SHA384
- ALPN, server did not agree to a protocol
- Server certificate:
- subject: CN=proteus.xxxx.com
- start date: Jan 26 20:50:40 2019 GMT
- expire date: Jan 23 20:50:40 2029 GMT
- common name: proteus.xxxx.com (matched)
- issuer: CN=proteus.xxxx.com
- SSL certificate verify ok.
GET / HTTP/1.1
Host: proteus.xxxx.com:3521
User-Agent: curl/7.63.0
Accept: /
- TLSv1.2 (IN), TLS alert, close notify (256):
- Empty reply from server
- Connection #0 to host proteus.xxxx.com left intact
/>
This is the Logstash logs after a restart of the logstash container.
<
27/01/2019 11:27:42[2019-01-27T11:27:42,071][WARN ][logstash.shutdownwatcher ] {"inflight_count"=>0, "stalling_thread_info"=>{"other"=>[{"thread_id"=>34, "name"=>"[main]<beats", "current_call"=>"[...]/vendor/bundle/jruby/2.3.0/gems/logstash-input-beats-5.0.10-java/lib/logstash/inputs/beats.rb:198:in run'"}, {"thread_id"=>35, "name"=>"[main]<tcp", "current_call"=>"uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/jopenssl23/openssl/ssl.rb:401:in
accept'"}, {"thread_id"=>36, "name"=>"[main]<beats", "current_call"=>"[...]/vendor/bundle/jruby/2.3.0/gems/logstash-input-beats-5.0.10-java/lib/logstash/inputs/beats.rb:198:in run'"}, {"thread_id"=>37, "name"=>"[main]<beats", "current_call"=>"[...]/vendor/bundle/jruby/2.3.0/gems/logstash-input-beats-5.0.10-java/lib/logstash/inputs/beats.rb:198:in
run'"}, {"thread_id"=>29, "name"=>nil, "current_call"=>"[...]/logstash-core/lib/logstash/util/wrapped_acked_queue.rb:171:in lock'"}, {"thread_id"=>30, "name"=>nil, "current_call"=>"[...]/logstash-core/lib/logstash/util/wrapped_acked_queue.rb:66:in
read_batch'"}]}}
27/01/2019 11:27:42[2019-01-27T11:27:42,115][ERROR][logstash.shutdownwatcher ] The shutdown process appears to be stalled due to busy or blocked plugins. Check the logs for more information.
27/01/2019 11:27:43[2019-01-27T11:27:43,571][ERROR][logstash.inputs.tcp ] TCP input server encountered error {:message=>"stream closed", :class=>"IOError", :backtrace=>["org/jruby/ext/socket/RubyTCPServer.java:157:in accept'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/jopenssl23/openssl/ssl.rb:401:in
accept'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-tcp-5.0.3-java/lib/logstash/inputs/tcp.rb:203:in run_ssl_server'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-tcp-5.0.3-java/lib/logstash/inputs/tcp.rb:152:in
run'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:516:in inputworker'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:509:in
block in start_input'"]}
27/01/2019 11:28:25Sending Logstash's logs to /usr/share/logstash/logs which is now configured via log4j2.properties
27/01/2019 11:28:25[2019-01-27T11:28:25,342][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"fb_apache", :directory=>"/usr/share/logstash/modules/fb_apache/configuration"}
27/01/2019 11:28:25[2019-01-27T11:28:25,368][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"netflow", :directory=>"/usr/share/logstash/modules/netflow/configuration"}
27/01/2019 11:28:26[2019-01-27T11:28:26,205][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
27/01/2019 11:28:27[2019-01-27T11:28:27,197][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"6.2.3"}
27/01/2019 11:28:28[2019-01-27T11:28:28,022][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
27/01/2019 11:28:38[2019-01-27T11:28:38,855][INFO ][logstash.pipeline ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>2, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
27/01/2019 11:28:39[2019-01-27T11:28:39,728][INFO ][logstash.outputs.rabbitmq] Connected to RabbitMQ at
27/01/2019 11:28:40[2019-01-27T11:28:40,791][INFO ][logstash.inputs.beats ] Beats inputs: Starting input listener {:address=>"0.0.0.0:3520"}
27/01/2019 11:28:40[2019-01-27T11:28:40,884][INFO ][logstash.inputs.tcp ] Starting tcp input listener {:address=>"0.0.0.0:3521", :ssl_enable=>"true"}
27/01/2019 11:28:42[2019-01-27T11:28:42,132][INFO ][logstash.inputs.beats ] Beats inputs: Starting input listener {:address=>"0.0.0.0:3522"}
27/01/2019 11:28:42[2019-01-27T11:28:42,143][INFO ][logstash.inputs.beats ] Beats inputs: Starting input listener {:address=>"0.0.0.0:3524"}
27/01/2019 11:28:42[2019-01-27T11:28:42,218][INFO ][logstash.pipeline ] Pipeline started succesfully {:pipeline_id=>"main", :thread=>"#<Thread:0x2a10079e run>"}
27/01/2019 11:28:42[2019-01-27T11:28:42,303][INFO ][org.logstash.beats.Server] Starting server on port: 3524
27/01/2019 11:28:42[2019-01-27T11:28:42,322][INFO ][org.logstash.beats.Server] Starting server on port: 3522
27/01/2019 11:28:42[2019-01-27T11:28:42,352][INFO ][org.logstash.beats.Server] Starting server on port: 3520
27/01/2019 11:28:42[2019-01-27T11:28:42,416][INFO ][logstash.inputs.syslog ] Starting syslog tcp listener {:address=>"0.0.0.0:1516"}
27/01/2019 11:28:42[2019-01-27T11:28:42,432][INFO ][logstash.inputs.syslog ] Starting syslog udp listener {:address=>"0.0.0.0:1516"}
27/01/2019 11:28:42[2019-01-27T11:28:42,609][INFO ][logstash.agent ] Pipelines running {:count=>1, :pipelines=>["main"]}
/>