Failed to connect to ip-myip port 80: Connection refused

I have installed elasticsearch 7 on Ubuntu 18. I have 3 nodes, all master eligible.

This is my configuration:

First server: 172.31.1.1

# /etc/elasticsearch/elasticsearch.yml
cluster.name: es-cluster
node.name: es-1

path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch

network.host: 172.31.1.1
http.port: 9200

discovery.seed_hosts: ["172.31.1.1", "172.31.1.2", "172.31.1.3"]
cluster.initial_master_nodes:["es-1", "es-2", "es-3"]

gateway.recover_after_nodes: 2
gateway.expected_nodes: 3

action.destructive_requires_name: true

Second server: 172.31.1.2

node.name: es-2
network.host: 172.31.1.2
# everything else same as first server

Third server: 172.31.1.3

node.name: es-3
network.host: 172.31.1.3
# everything else same as first server

This is /etc/default/elasticsearch

# /etc/default/elasticsearch
JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-amd64
ES_PATH_CONF=/etc/elasticsearch
RESTART_ON_UPGRADE=true
ES_STARTUP_SLEEP_TIME=5
MAX_OPEN_FILES=65535

START_DAEMON=true
ES_HEAP_SIZE=2g
# I am not sure if I should add these or not?
# LOG_DIR=/var/log/elasticsearch
# DATA_DIR=/var/lib/elasticsearch
# WORK_DIR=/tmp/elasticsearch
# CONF_DIR=/etc/elasticsearch
# CONF_FILE=/etc/elasticsearch/elasticsearch.yml

# I have tried adding these variables but got error saying these are no longer in use
# ES_USER=elasticsearch
# ES_GROUP=elasticsearch

And the file: /usr/lib/systemd/system/elasticsearch.service

# /usr/lib/systemd/system/elasticsearch.service

# some more config not shown... I have not changed this file... all default values
[Service]
RuntimeDirectory=elasticsearch
PrivateTmp=true
Environment=ES_HOME=/usr/share/elasticsearch
Environment=ES_PATH_CONF=/etc/elasticsearch
Environment=PID_DIR=/var/run/elasticsearch
EnvironmentFile=-/etc/default/elasticsearch
WorkingDirectory=/usr/share/elasticsearch

User=elasticsearch
Group=elasticsearch

ExecStart=/usr/share/elasticsearch/bin/elasticsearch -p ${PID_DIR}/elasticsearch.pid

I have changed the owner of the following files to elasticsearch:elasticsearch

sudo chown elasticsearch:elasticsearch -R /usr/share/elasticsearch
sudo chown elasticsearch:elasticsearch -R /var/log/elasticsearch
sudo chown elasticsearch:elasticsearch -R /var/lib/elasticsearch
sudo chown elasticsearch:elasticsearch -R /etc/default/elasticsearch
sudo chown elasticsearch:elasticsearch -R /etc/elasticsearch

Now I start the first node (only the first node)

sudo /bin/systemctl daemon-reload
sudo /bin/systemctl enable elasticsearch.service
sudo systemctl start elasticsearch.service

First issue is that nothing is written to log file

If I check the status of elasticsearch I have:

Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled)
   Active: active (running) since Sun 2019-05-26 11:12:25 AEST; 46min ago
     Docs: http://www.elastic.co
 Main PID: 5014 (java)
    Tasks: 40 (limit: 4915)
   CGroup: /system.slice/elasticsearch.service
           ├─5014 /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java ...
           └─5088 /usr/share/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bin/controller

[INFO ][o.e.d.DiscoveryModule    ] [ip-172-31-1-1] using discovery type [zen] and seed h
[INFO ][o.e.n.Node               ] [ip-172-31-1-1] initialized
[INFO ][o.e.n.Node               ] [ip-172-31-1-1] starting ...
[INFO ][o.e.t.TransportService   ] [ip-172-31-1-1] publish_address {172.31.1.1:9300},
[INFO ][o.e.b.BootstrapChecks    ] [ip-172-31-1-1] bound or publishing to a non-loopback
[INFO ][o.e.c.c.Coordinator      ] [ip-172-31-1-1] cluster UUID [...]
[INFO ][o.e.c.s.MasterService    ] [ip-172-31-1-1] elected-as-master ([1] nodes joined)[
[INFO ][o.e.c.s.ClusterApplierService] [ip-172-31-1-1] master node changed {previous [],
[INFO ][o.e.h.AbstractHttpServerTransport] [ip-172-31-1-1] publish_address {172.31.1.1
[INFO ][o.e.n.Node               ] [ip-172-31-1-1] started

I have no idea why it is saying:

publish_address {172.31.1.1:9300} // shouldn't be 9200?

If I check the health of cluster

curl http://ip-172-31-1-1:9200/_cat/health
somenumber 02:05:56 es-cluster red 1 1 0 0 0 0 0 0 - NaN%

If I try:

curl http://ip-172-31-1-1/:9200/_cat/nodes
curl: (7) Failed to connect to ip-172-31-1-1 port 80: Connection refused

If I try:

journalctl -f

[INFO ][o.e.c.s.ClusterApplierService] [ip-172-31-11-1] master node changed {previous [], current [{ip-172-31-1-1}{...}{...}{172.31.1.1}{172.31.1.1:9300}{ml.machine_memory=..., xpack.installed=true, ml.max_open_jobs=20}]}, term: 27, version: 45, reason: Publication{term=27, version=45}
[INFO ][o.e.h.AbstractHttpServerTransport] [ip-172-31-1-1] publish_address {172.31.1.1:9200}, bound_addresses {172.31.1.1:9200}
[INFO ][o.e.n.Node               ] [ip-172-31-1-1] started

Take out the / before the port#

Did you install using the DEB?

@rugnel, thanks for your comment. I updated the question... I have commented out port 9200 and I realized I was reading the wring log file... I have included my log exception.

@warkolm: thanks a lot. I have updated the question. I removed port : 9200 from
elasticsearch.yml, because I believe it should have been 9300? So the port is now comment out.

I have also included the log file (I was reading the wrong log file)

This is how I installed elasticseach:

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
sudo add-apt-repository "deb https://artifacts.elastic.co/packages/7.x/apt stable main"
sudo apt-get update
sudo apt-get install elasticsearch

Which port?

So why did you do the chown?

in /etc/elasticsearch/elasticsearch.yml

I had:

http.port: 9200

Now I removed it... I think that was one problem...

As I was not able to read the log file (I was reading the wrong log files) I was not sure what I am doing... I followed this question: https://stackoverflow.com/questions/40766301/run-elastic-search-as-root-user/50617708#50617708

I made 2 modifications to: /etc/elasticsearch/elasticsearch.yml

network.host: 172.31.1.1
# http.port: 9200 <-- commented

# only included other servers in discovery.seed_hosts 
discovery.seed_hosts: ["172.31.1.2", "172.31.1.3"]  

Then I kept first node running. stopped elasticsearch on second and third nodes, deleted their data path (content of /var/lib/elasticsearch in my case) and restarted elasticsearch on second and third node and they are now joining the cluster.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.