Logstash unable to reach Elasticsearch: port issues

[rahulnama]

Hello Team

I deployed Elasticsearch on Kubernetes and it's working fine. Also, I created an ingress so it's redirecting from 80 => 9200. But logstash is looking for 9200 so connection is failing.

However, if I try http://elasticsearch.phoenix-alfresco.10.85.77.116.xip.io:9200 it's giving me error but http://elasticsearch.phoenix-alfresco.10.85.77.116.xip.io is working fine as we can see in the above figure

logstash.conf: -- elasticsearch host setting

xpack.monitoring.elasticsearch.hosts: [ "http://elasticsearch.phoenix-alfresco.10.85.77.116.xip.io" ]

logstash logs:

[2020-07-14T19:51:57,137][WARN ][logstash.outputs.elasticsearch][main] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://elastic:xxxxxx@elasticsearch.phoenix-alfresco.10.85.77.116.xip.io:9200/", :error_type=>LogStash::Outputs::Elasticsearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [http://elastic:xxxxxx@elasticsearch.phoenix-alfresco.10.85.77.116.xip.io:9200/][Manticore::SocketException] Connection refused (Connection refused)"}

How to make logstash look for http://elasticsearch.phoenix-alfresco.10.85.77.116.xip.io instead of http://elasticsearch.phoenix-alfresco.10.85.77.116.xip.io:9200

please let me know.

Thank you

-Rahul
`

[Badger]

@rahulnama

url=>"http://elastic:xxxxxx@elasticsearch.phoenix-alfresco.10.85.77.116.xip.io:9200/"

If that is the url you configured in the hosts option of the elasticsearch output then just change 9200 to 80.

[rahulnama]

Hi @Badger

Thank you the response.

I tried that as well in logstash config. No matter what logstash is looking for :9200.

[Badger]

Is the elasticsearch output looking for 9200, or is xpack monitoring or the licence checker looking for 9200?

[rahulnama]

Hi @Badger
when I update conf with :80 instead of :9200

at first it connected. here is the log

[2020-07-14T20:18:13,047][WARN ][logstash.licensechecker.licensereader] Restored connection to ES instance {:url=>"http://elastic:xxxxxx@elasticsearch.phoenix-alfresco.10.85.77.116.xip.io:80/"}

and then immediately I see this updating. log:

[2020-07-14T20:18:17,864][INFO ][logstash.outputs.elasticsearch][main] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://elastic:xxxxxx@elasticsearch.phoenix-alfresco.10.85.77.116.xip.io:9200/]}}

I believe this is x-pack issue. Any suggestions here ?

[rahulnama]

Hi @Badger

As you said changing the elasticsearch output in logstash.yml file to http://elasticsearch_url:80 solved this issue. So far I have been updating it in logstash.conf file.

Thank you

Best
Rahul