Logstash Elasticsearch Connection Refused

Elastic Stack Elasticsearch
1

I have 2 machine with both have logstash.
When I try to connect to Elasticsearch, it results to Connection Refused in next couple of minutes.
I also tried to open 2 ports in and port forward it to my es, but still the same.
What would be the cause for this one?

TIA

2

Hi,

which version are you waorking for logstash and elasticsearch?
and is there any configuration file you have do changes?

3

Can you e.g. curl the ES API from the Logstash machine? That should show you if there

Logstash logs are pretty good for finding the reason why Logstash thinks it can't connect.

Also, do you have x-pack installed?

Your Logstash output configuration would be good to post.

4

Hi @poojagupta,

I'm currently running on versiokn 5.5.2 on both elasticsearch and logstash.
The files i've changed are the config files of both es and ls.

5

Hi @A_B,

nope. I can't connect to es machine from the logstash machine. This maybe due that the port was blocked.
I also tried to use netstat command and it stated that the port for elasticsearch is on CLOSE_WAIT state.

Right now I dont have x-pack installed in my system.

[2017-09-07T15:49:44,050][ERROR][logstash.outputs.elasticsearch] Attempted to send a bulk request to elasticsearch, but no there are no living connections in the connection pool. Perhaps Elasticsearch is unreachable or down? {:error_message=>"No Available connections", :class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError", :will_retry_in_seconds=>64}
[2017-09-07T15:49:45,442][WARN ][logstash.shutdownwatcher ] {"inflight_count"=>497, "stalling_thread_info"=>{"other"=>[{"thread_id"=>20, "name"=>"[main]>worker0", "current_call"=>"[...]/vendor/bundle/jruby/1.9/gems/stud-0.0.23/lib/stud/interval.rb:89:in `sleep'"}, {"thread_id"=>21, "name"=>"[main]>worker1", "current_call"=>"[...]/vendor/bundle/jruby/1.9/gems/stud-0.0.23/lib/stud/interval.rb:89:in `sleep'"}, {"thread_id"=>22, "name"=>"[main]>worker2", "current_call"=>"[...]/vendor/bundle/jruby/1.9/gems/stud-0.0.23/lib/stud/interval.rb:89:in `sleep'"}, {"thread_id"=>23, "name"=>"[main]>worker3", "current_call"=>"[...]/vendor/bundle/jruby/1.9/gems/stud-0.0.23/lib/stud/interval.rb:89:in `sleep'"}]}}
[2017-09-07T15:49:47,672][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://192.168.200.2:10014/, :path=>"/"}
[2017-09-07T15:50:38,120][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://192.168.200.2:10014/, :path=>"/"}
[2017-09-07T15:50:38,163][WARN ][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://192.168.200.2:10014/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [http://192.168.200.2:10014/][Manticore::SocketException] Connection refused"}
[2017-09-07T15:50:40,438][WARN ][logstash.shutdownwatcher ] {"inflight_count"=>497, "stalling_thread_info"=>{"other"=>[{"thread_id"=>20, "name"=>"[main]>worker0", "current_call"=>"[...]/vendor/bundle/jruby/1.9/gems/stud-0.0.23/lib/stud/interval.rb:89:in `sleep'"}, {"thread_id"=>21, "name"=>"[main]>worker1", "current_call"=>"[...]/vendor/bundle/jruby/1.9/gems/stud-0.0.23/lib/stud/interval.rb:89:in `sleep'"}, {"thread_id"=>22, "name"=>"[main]>worker2", "current_call"=>"[...]/vendor/bundle/jruby/1.9/gems/stud-0.0.23/lib/stud/interval.rb:89:in `sleep'"}, {"thread_id"=>23, "name"=>"[main]>worker3", "current_call"=>"[...]/vendor/bundle/jruby/1.9/gems/stud-0.0.23/lib/stud/interval.rb:89:in `sleep'"}]}}
6

Hi @jogoinar10

Looking at logstash output, i observed that logstash is unable to communicate with elasticsearch due to port.
Firstly open the 9200 and 9300 port. using firewalld command you can enable and open the port.
then start the both services (logstash and elasticsearch) .
Please send the elasticsearch logs also for more understanding.

7

on my es server, only port 9200 is open.
why do i need to open port 9300? hmmm... :thinking:

8

Hi @jogoinar10

9300 port is also used for nodes communication.
In Elasticsearch , 9200 -9300 range are used for binding host and communicate between nodes in the cluster also.
In elasticsearch one port is binded for HTTP and REST API.

It'll try the lowest available port first, and if it is already taken, try the next. If you run a single node on your machine, it'll only bind to 9200 and 9300.
so it can be reason for unableing to communicate with it.

1 Like
9

Thanks for the info @poojagupta.
Gonna try to open the port 9300.

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.