Failed to connect to localhost port 9200: Connection refused

I am trying to install Elasticsearch 8.4.0 and kibana on Rocky Linux 8.6 using this article.

Both elasticsearch and kibana are active and running but running this cmd:

curl -X GET 'http://localhost:9200'

I get
curl: (7) Failed to connect to localhost port 9200: Connection refused

I also ran netstat and it doesn't show port 9200 or 5601 listening

Logs of /var/log/elasticsearch/elasticsearch.log

[2022-08-25T17:35:14,039][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-monitoring]
[2022-08-25T17:35:14,039][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-ql]
[2022-08-25T17:35:14,040][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-rollup]
[2022-08-25T17:35:14,040][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-security]
[2022-08-25T17:35:14,040][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-shutdown]
[2022-08-25T17:35:14,040][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-sql]
[2022-08-25T17:35:14,040][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-stack]
[2022-08-25T17:35:14,040][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-text-structure]
[2022-08-25T17:35:14,040][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-voting-only-node]
[2022-08-25T17:35:14,041][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-watcher]
[2022-08-25T17:35:14,041][INFO ][o.e.p.PluginsService     ] [node-1] no plugins loaded
[2022-08-25T17:35:18,454][INFO ][o.e.e.NodeEnvironment    ] [node-1] using [1] data paths, mounts [[/ (/dev/sda1)]], net usable_space [6.4gb], net total_space [19.9gb], types [xfs]
[2022-08-25T17:35:18,454][INFO ][o.e.e.NodeEnvironment    ] [node-1] heap size [1.4gb], compressed ordinary object pointers [true]
[2022-08-25T17:35:18,586][INFO ][o.e.n.Node               ] [node-1] node name [node-1], node ID [_vCEMBokSMaMyYLk4msUog], cluster name [elasticsearch], roles [data_hot, transform, data_content, data_warm, master, remote_cluster_client, data, data_cold, ingest, data_frozen, ml]
[2022-08-25T17:35:25,111][INFO ][o.e.x.s.Security         ] [node-1] Security is enabled
[2022-08-25T17:35:25,440][INFO ][o.e.x.s.a.s.FileRolesStore] [node-1] parsed [0] roles from file [/etc/elasticsearch/roles.yml]
[2022-08-25T17:35:26,022][INFO ][o.e.x.s.InitialNodeSecurityAutoConfiguration] [node-1] Auto-configuration will not generate a password for the elastic built-in superuser, as we cannot  determine if there is a terminal attached to the elasticsearch process. You can use the `bin/elasticsearch-reset-password` tool to set the password for the elastic user.
[2022-08-25T17:35:26,407][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [node-1] [controller/1940450] [Main.cc@123] controller (64 bit): Version 8.4.0 (Build 70eefc6b414c79) Copyright (c) 2022 Elasticsearch BV
[2022-08-25T17:35:27,272][INFO ][o.e.t.n.NettyAllocator   ] [node-1] creating NettyAllocator with the following configs: [name=elasticsearch_configured, chunk_size=1mb, suggested_max_allocation_size=1mb, factors={es.unsafe.use_netty_default_chunk_and_page_size=false, g1gc_enabled=true, g1gc_region_size=4mb}]
[2022-08-25T17:35:27,308][INFO ][o.e.i.r.RecoverySettings ] [node-1] using rate limit [40mb] with [default=40mb, read=0b, write=0b, max=0b]
[2022-08-25T17:35:27,348][INFO ][o.e.d.DiscoveryModule    ] [node-1] using discovery type [multi-node] and seed hosts providers [settings]
[2022-08-25T17:35:29,701][INFO ][o.e.n.Node               ] [node-1] initialized
[2022-08-25T17:35:29,702][INFO ][o.e.n.Node               ] [node-1] starting ...
[2022-08-25T17:35:29,808][INFO ][o.e.x.s.c.f.PersistentCache] [node-1] persistent cache index loaded
[2022-08-25T17:35:29,808][INFO ][o.e.x.d.l.DeprecationIndexingComponent] [node-1] deprecation component started
[2022-08-25T17:35:30,147][INFO ][o.e.t.TransportService   ] [node-1] publish_address {127.0.0.1:9300}, bound_addresses {127.0.0.1:9300}
[2022-08-25T17:35:30,872][WARN ][o.e.c.c.ClusterBootstrapService] [node-1] this node is locked into cluster UUID [dkdCh8ORSw69OXZKEqWKwg] but [cluster.initial_master_nodes] is set to [loganalysisclass1.novalocal]; remove this setting to avoid possible data loss caused by subsequent cluster bootstrap attempts
[2022-08-25T17:35:31,191][INFO ][o.e.c.s.MasterService    ] [node-1] elected-as-master ([1] nodes joined)[_FINISH_ELECTION_, {node-1}{_vCEMBokSMaMyYLk4msUog}{t7EV2_RCQTmGxOetLmjIPQ}{node-1}{127.0.0.1}{127.0.0.1:9300}{cdfhilmrstw} completing election], term: 11, version: 89, delta: master node changed {previous [], current [{node-1}{_vCEMBokSMaMyYLk4msUog}{t7EV2_RCQTmGxOetLmjIPQ}{node-1}{127.0.0.1}{127.0.0.1:9300}{cdfhilmrstw}]}
[2022-08-25T17:35:31,364][INFO ][o.e.c.s.ClusterApplierService] [node-1] master node changed {previous [], current [{node-1}{_vCEMBokSMaMyYLk4msUog}{t7EV2_RCQTmGxOetLmjIPQ}{node-1}{127.0.0.1}{127.0.0.1:9300}{cdfhilmrstw}]}, term: 11, version: 89, reason: Publication{term=11, version=89}
[2022-08-25T17:35:31,423][INFO ][o.e.r.s.FileSettingsService] [node-1] starting file settings watcher ...
[2022-08-25T17:35:31,519][INFO ][o.e.r.s.FileSettingsService] [node-1] file settings service up and running [tid=46]
[2022-08-25T17:35:31,528][INFO ][o.e.h.AbstractHttpServerTransport] [node-1] publish_address {10.0.98.168:9200}, bound_addresses {[::]:9200}
[2022-08-25T17:35:31,528][INFO ][o.e.n.Node               ] [node-1] started {node-1}{_vCEMBokSMaMyYLk4msUog}{t7EV2_RCQTmGxOetLmjIPQ}{node-1}{127.0.0.1}{127.0.0.1:9300}{cdfhilmrstw}{ml.max_jvm_size=1564475392, ml.allocated_processors=1, xpack.installed=true, ml.machine_memory=3129405440}
[2022-08-25T17:35:31,757][INFO ][o.e.l.LicenseService     ] [node-1] license [0483c333-2106-4c53-956e-1fac5ecdec59] mode [basic] - valid
[2022-08-25T17:35:31,758][INFO ][o.e.x.s.a.Realms         ] [node-1] license mode is [basic], currently licensed security realms are [reserved/reserved,file/default_file,native/default_native]
[2022-08-25T17:35:31,780][INFO ][o.e.g.GatewayService     ] [node-1] recovered [1] indices into cluster_state
[2022-08-25T17:35:32,056][ERROR][o.e.i.g.GeoIpDownloader  ] [node-1] exception during geoip databases update
org.elasticsearch.ElasticsearchException: not all primary shards of [.geoip_databases] index are active
        at org.elasticsearch.ingest.geoip.GeoIpDownloader.updateDatabases(GeoIpDownloader.java:134) ~[?:?]
        at org.elasticsearch.ingest.geoip.GeoIpDownloader.runDownloader(GeoIpDownloader.java:274) ~[?:?]
        at org.elasticsearch.ingest.geoip.GeoIpDownloaderTaskExecutor.nodeOperation(GeoIpDownloaderTaskExecutor.java:102) ~[?:?]
        at org.elasticsearch.ingest.geoip.GeoIpDownloaderTaskExecutor.nodeOperation(GeoIpDownloaderTaskExecutor.java:48) ~[?:?]
        at org.elasticsearch.persistent.NodePersistentTasksExecutor$1.doRun(NodePersistentTasksExecutor.java:42) ~[elasticsearch-8.4.0.jar:?]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:769) ~[elasticsearch-8.4.0.jar:?]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) ~[elasticsearch-8.4.0.jar:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
        at java.lang.Thread.run(Thread.java:833) ~[?:?]
[2022-08-25T17:35:33,003][INFO ][o.e.c.r.a.AllocationService] [node-1] current.health="GREEN" message="Cluster health status changed from [RED] to [GREEN] (reason: [shards started [[.geoip_databases][0]]])." previous.health="RED" reason="shards started [[.geoip_databases][0]]"
[2022-08-25T17:35:33,421][INFO ][o.e.i.g.DatabaseNodeService] [node-1] successfully loaded geoip database file [GeoLite2-Country.mmdb]
[2022-08-25T17:35:33,515][INFO ][o.e.i.g.DatabaseNodeService] [node-1] successfully loaded geoip database file [GeoLite2-ASN.mmdb]
[2022-08-25T17:35:34,489][INFO ][o.e.i.g.DatabaseNodeService] [node-1] successfully loaded geoip database file [GeoLite2-City.mmdb]
[2022-08-25T17:36:32,520][WARN ][o.e.t.ThreadPool         ] [node-1] execution of [org.elasticsearch.cluster.InternalClusterInfoService$RefreshScheduler$$Lambda$6355/0x0000000801bd27c0@7db894b8] took [28704ms] which is above the warn threshold of [5000ms]

Please assist.

Elasticsearch is listening to a local address assigned to the container, so try 10.0.98.168:9200.

Sharing your elasticsearch.yml would help diagnose issues.

Thanks, I was able to fix that but elasticsearch keeps stopping after running for a while. As per my research it looks like it has something to do with bootstrap checks but I am not able to figure out how to fix them.
Here are the logs from elacticsearch.log and elasticsearch.yml file.

[2022-08-26T18:12:49,030][INFO ][o.e.p.PluginsService     ] [loganalysisclass1.novalocal] loaded module [x-pack-graph]
[2022-08-26T18:12:49,030][INFO ][o.e.p.PluginsService     ] [loganalysisclass1.novalocal] loaded module [x-pack-identity-provider]
[2022-08-26T18:12:49,030][INFO ][o.e.p.PluginsService     ] [loganalysisclass1.novalocal] loaded module [x-pack-ilm]
[2022-08-26T18:12:49,030][INFO ][o.e.p.PluginsService     ] [loganalysisclass1.novalocal] loaded module [x-pack-logstash]
[2022-08-26T18:12:49,030][INFO ][o.e.p.PluginsService     ] [loganalysisclass1.novalocal] loaded module [x-pack-ml]
[2022-08-26T18:12:49,031][INFO ][o.e.p.PluginsService     ] [loganalysisclass1.novalocal] loaded module [x-pack-monitoring]
[2022-08-26T18:12:49,031][INFO ][o.e.p.PluginsService     ] [loganalysisclass1.novalocal] loaded module [x-pack-ql]
[2022-08-26T18:12:49,031][INFO ][o.e.p.PluginsService     ] [loganalysisclass1.novalocal] loaded module [x-pack-rollup]
[2022-08-26T18:12:49,031][INFO ][o.e.p.PluginsService     ] [loganalysisclass1.novalocal] loaded module [x-pack-security]
[2022-08-26T18:12:49,031][INFO ][o.e.p.PluginsService     ] [loganalysisclass1.novalocal] loaded module [x-pack-shutdown]
[2022-08-26T18:12:49,031][INFO ][o.e.p.PluginsService     ] [loganalysisclass1.novalocal] loaded module [x-pack-sql]
[2022-08-26T18:12:49,032][INFO ][o.e.p.PluginsService     ] [loganalysisclass1.novalocal] loaded module [x-pack-stack]
[2022-08-26T18:12:49,032][INFO ][o.e.p.PluginsService     ] [loganalysisclass1.novalocal] loaded module [x-pack-text-structure]
[2022-08-26T18:12:49,032][INFO ][o.e.p.PluginsService     ] [loganalysisclass1.novalocal] loaded module [x-pack-voting-only-node]
[2022-08-26T18:12:49,032][INFO ][o.e.p.PluginsService     ] [loganalysisclass1.novalocal] loaded module [x-pack-watcher]
[2022-08-26T18:12:49,032][INFO ][o.e.p.PluginsService     ] [loganalysisclass1.novalocal] no plugins loaded
[2022-08-26T18:12:54,921][INFO ][o.e.e.NodeEnvironment    ] [loganalysisclass1.novalocal] using [1] data paths, mounts [[/ (/dev/sda1)]], net usable_space [6.4gb], net total_space [19.9gb], types [xfs]
[2022-08-26T18:12:54,934][INFO ][o.e.e.NodeEnvironment    ] [loganalysisclass1.novalocal] heap size [1.4gb], compressed ordinary object pointers [true]
[2022-08-26T18:12:55,172][INFO ][o.e.n.Node               ] [loganalysisclass1.novalocal] node name [loganalysisclass1.novalocal], node ID [_vCEMBokSMaMyYLk4msUog], cluster name [elasticsearch], roles [data_frozen, ingest, data_cold, data, remote_cluster_client, master, data_warm, data_content, transform, data_hot, ml]
[2022-08-26T18:13:05,200][INFO ][o.e.x.s.Security         ] [loganalysisclass1.novalocal] Security is disabled
[2022-08-26T18:13:05,803][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [loganalysisclass1.novalocal] [controller/2003902] [Main.cc@123] controller (64 bit): Version 8.4.0 (Build 70eefc6b414c79) Copyright (c) 2022 Elasticsearch BV
[2022-08-26T18:13:09,665][INFO ][o.e.t.n.NettyAllocator   ] [loganalysisclass1.novalocal] creating NettyAllocator with the following configs: [name=elasticsearch_configured, chunk_size=1mb, suggested_max_allocation_size=1mb, factors={es.unsafe.use_netty_default_chunk_and_page_size=false, g1gc_enabled=true, g1gc_region_size=4mb}]
[2022-08-26T18:13:10,722][INFO ][o.e.i.r.RecoverySettings ] [loganalysisclass1.novalocal] using rate limit [40mb] with [default=40mb, read=0b, write=0b, max=0b]
[2022-08-26T18:13:12,952][INFO ][o.e.d.DiscoveryModule    ] [loganalysisclass1.novalocal] using discovery type [multi-node] and seed hosts providers [settings]
[2022-08-26T19:46:24,557][ERROR][o.e.b.Elasticsearch      ] [loganalysisclass1.novalocal] fatal exception while booting Elasticsearch
java.nio.file.AccessDeniedException: /var/run/elasticsearch
        at sun.nio.fs.UnixException.translateToIOException(UnixException.java:90) ~[?:?]
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106) ~[?:?]
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) ~[?:?]
        at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:397) ~[?:?]
        at java.nio.file.Files.createDirectory(Files.java:700) ~[?:?]
        at java.nio.file.Files.createAndCheckIsDirectory(Files.java:807) ~[?:?]
        at java.nio.file.Files.createDirectories(Files.java:793) ~[?:?]
        at org.elasticsearch.bootstrap.Elasticsearch.initPidFile(Elasticsearch.java:381) ~[elasticsearch-8.4.0.jar:?]
        at org.elasticsearch.bootstrap.Elasticsearch.initPhase2(Elasticsearch.java:156) ~[elasticsearch-8.4.0.jar:?]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:66) ~[elasticsearch-8.4.0.jar:?]
[2022-08-26T20:12:14,007][ERROR][o.e.b.Elasticsearch      ] [loganalysisclass1.novalocal] fatal exception while booting Elasticsearch
java.nio.file.AccessDeniedException: /var/run/elasticsearch
        at sun.nio.fs.UnixException.translateToIOException(UnixException.java:90) ~[?:?]
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106) ~[?:?]
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) ~[?:?]
        at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:397) ~[?:?]
        at java.nio.file.Files.createDirectory(Files.java:700) ~[?:?]
        at java.nio.file.Files.createAndCheckIsDirectory(Files.java:807) ~[?:?]
        at java.nio.file.Files.createDirectories(Files.java:793) ~[?:?]
        at org.elasticsearch.bootstrap.Elasticsearch.initPidFile(Elasticsearch.java:381) ~[elasticsearch-8.4.0.jar:?]
        at org.elasticsearch.bootstrap.Elasticsearch.initPhase2(Elasticsearch.java:156) ~[elasticsearch-8.4.0.jar:?]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:66) ~[elasticsearch-8.4.0.jar:?]
[2022-08-26T20:28:32,283][ERROR][o.e.b.Elasticsearch      ] [loganalysisclass1.novalocal] fatal exception while booting Elasticsearch
java.nio.file.AccessDeniedException: /var/run/elasticsearch
        at sun.nio.fs.UnixException.translateToIOException(UnixException.java:90) ~[?:?]
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106) ~[?:?]
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) ~[?:?]
        at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:397) ~[?:?]
        at java.nio.file.Files.createDirectory(Files.java:700) ~[?:?]
        at java.nio.file.Files.createAndCheckIsDirectory(Files.java:807) ~[?:?]
        at java.nio.file.Files.createDirectories(Files.java:793) ~[?:?]
        at org.elasticsearch.bootstrap.Elasticsearch.initPidFile(Elasticsearch.java:381) ~[elasticsearch-8.4.0.jar:?]
        at org.elasticsearch.bootstrap.Elasticsearch.initPhase2(Elasticsearch.java:156) ~[elasticsearch-8.4.0.jar:?]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:66) ~[elasticsearch-8.4.0.jar:?]

elasticsearch.yml

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
#node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/elasticsearch
#
# Path to log files:
#
path.logs: /var/log/elasticsearch
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# By default Elasticsearch is only accessible on localhost. Set a different
# address here to expose this node on the network:
#
network.host: 10.0.98.168
#
# By default Elasticsearch listens for HTTP traffic on the first free port it
# finds starting at 9200. Set a specific HTTP port here:
#
#http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
#cluster.initial_master_nodes: ["node-1", "node-2"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# --------------------------------- Readiness ----------------------------------
#
# Enable an unauthenticated TCP readiness endpoint on localhost
#
#readiness.port: 9399
#
# ---------------------------------- Various -----------------------------------
#
# Allow wildcard deletion of indices:
#
#action.destructive_requires_name: false
action.auto_create_index: .monitoring*,.watches,.triggered_watches,.watcher-history*,.ml*

#----------------------- BEGIN SECURITY AUTO CONFIGURATION -----------------------
#
# The following settings, TLS certificates, and keys have been automatically
# generated to configure Elasticsearch security features on 24-08-2022 16:56:04
#
# --------------------------------------------------------------------------------

# Enable security features
xpack.security.enabled: false

xpack.security.enrollment.enabled: false

# Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents
xpack.security.http.ssl:
  enabled: false
  keystore.path: certs/http.p12

# Enable encryption and mutual authentication between cluster nodes
xpack.security.transport.ssl:
  enabled: false
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12
# Create a new cluster with the current node only
# Additional nodes can still join the cluster later
cluster.initial_master_nodes: ["loganalysisclass1.novalocal"]

# Allow HTTP API connections from anywhere
# Connections are encrypted and require user authentication
http.host: 0.0.0.0

# Allow other nodes to join the cluster from anywhere
# Connections are encrypted and mutually authenticated
#transport.host: 0.0.0.0

What does your elasticsearch.yml look like?

I have attached it above. Pasting it below again.


# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
#node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/elasticsearch
#
# Path to log files:
#
path.logs: /var/log/elasticsearch
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# By default Elasticsearch is only accessible on localhost. Set a different
# address here to expose this node on the network:
#
network.host: 10.0.98.168
#
# By default Elasticsearch listens for HTTP traffic on the first free port it
# finds starting at 9200. Set a specific HTTP port here:
#
#http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
#cluster.initial_master_nodes: ["node-1", "node-2"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# --------------------------------- Readiness ----------------------------------
#
# Enable an unauthenticated TCP readiness endpoint on localhost
#
#readiness.port: 9399
#
# ---------------------------------- Various -----------------------------------
#
# Allow wildcard deletion of indices:
#
#action.destructive_requires_name: false
action.auto_create_index: .monitoring*,.watches,.triggered_watches,.watcher-history*,.ml*

#----------------------- BEGIN SECURITY AUTO CONFIGURATION -----------------------
#
# The following settings, TLS certificates, and keys have been automatically
# generated to configure Elasticsearch security features on 24-08-2022 16:56:04
#
# --------------------------------------------------------------------------------

# Enable security features
xpack.security.enabled: false

xpack.security.enrollment.enabled: false

# Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents
xpack.security.http.ssl:
  enabled: false
  keystore.path: certs/http.p12

# Enable encryption and mutual authentication between cluster nodes
xpack.security.transport.ssl:
  enabled: false
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12
# Create a new cluster with the current node only
# Additional nodes can still join the cluster later
cluster.initial_master_nodes: ["loganalysisclass1.novalocal"]

# Allow HTTP API connections from anywhere
# Connections are encrypted and require user authentication
http.host: 0.0.0.0

# Allow other nodes to join the cluster from anywhere
# Connections are encrypted and mutually authenticated
#transport.host: 0.0.0.0

That's why. It will only listen on that IP, and that IP is not localhost.
You may want to consider changing it to 0.0.0.0, which will listen to all interfaces.

Failed to connect to localhost on port 9200 is resolved. I was able to reach elasticsearch using curl command.

The new issue here is elasticsearch fails to start up or just crashes after few minutes. As per my research, this is some memory related issue but I can't find a solution. Please assist.

Redirecting to /bin/systemctl status elasticsearch.service
â—Ź elasticsearch.service - Elasticsearch
   Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: disabled)
   Active: failed (Result: timeout) since Tue 2022-08-30 18:40:36 UTC; 3min 14s ago
     Docs: https://www.elastic.co
  Process: 2338753 ExecStart=/usr/share/elasticsearch/bin/systemd-entrypoint -p ${PID_DIR}/elasticsearch.pid --quiet (code=exited, status=137)
 Main PID: 2338753 (code=exited, status=137)

Aug 30 18:39:16 loganalysisclass1.novalocal systemd[1]: Starting Elasticsearch...
Aug 30 18:40:31 loganalysisclass1.novalocal systemd[1]: elasticsearch.service: start operation timed out. Terminating.
Aug 30 18:40:36 loganalysisclass1.novalocal systemd-entrypoint[2338753]: ERROR: Elasticsearch exited unexpectedly
Aug 30 18:40:36 loganalysisclass1.novalocal systemd[1]: elasticsearch.service: Main process exited, code=exited, status=137/n/a
Aug 30 18:40:36 loganalysisclass1.novalocal systemd[1]: elasticsearch.service: Failed with result 'timeout'.
Aug 30 18:40:36 loganalysisclass1.novalocal systemd[1]: Failed to start Elasticsearch.

Per the docs... look at / show us the logs

sudo journalctl --unit elasticsearch

Here:

journalctl --unit elasticsearch
-- Logs begin at Thu 2022-07-28 14:49:07 UTC, end at Tue 2022-08-30 22:25:12 UTC. --
Aug 30 13:42:55 loganalysisclass1.novalocal systemd[1]: Starting Elasticsearch...
Aug 30 13:44:11 loganalysisclass1.novalocal systemd[1]: elasticsearch.service: start operation timed out. Terminating.
Aug 30 13:45:07 loganalysisclass1.novalocal systemd[1]: elasticsearch.service: Failed with result 'timeout'.
Aug 30 13:45:07 loganalysisclass1.novalocal systemd[1]: Failed to start Elasticsearch.
Aug 30 13:45:57 loganalysisclass1.novalocal systemd[1]: Starting Elasticsearch...
Aug 30 13:47:14 loganalysisclass1.novalocal systemd[1]: elasticsearch.service: start operation timed out. Terminating.
Aug 30 13:47:20 loganalysisclass1.novalocal systemd-entrypoint[2326444]: ERROR: Elasticsearch exited unexpectedly
Aug 30 13:47:20 loganalysisclass1.novalocal systemd[1]: elasticsearch.service: Main process exited, code=exited, status=137/n/a
Aug 30 13:47:20 loganalysisclass1.novalocal systemd[1]: elasticsearch.service: Failed with result 'timeout'.
Aug 30 13:47:20 loganalysisclass1.novalocal systemd[1]: Failed to start Elasticsearch.
Aug 30 18:29:17 loganalysisclass1.novalocal systemd[1]: Starting Elasticsearch...
Aug 30 18:30:33 loganalysisclass1.novalocal systemd[1]: elasticsearch.service: start operation timed out. Terminating.
Aug 30 18:30:33 loganalysisclass1.novalocal systemd-entrypoint[2337992]: ERROR: Elasticsearch exited unexpectedly
Aug 30 18:30:33 loganalysisclass1.novalocal systemd[1]: elasticsearch.service: Failed with result 'timeout'.
Aug 30 18:30:33 loganalysisclass1.novalocal systemd[1]: Failed to start Elasticsearch.
Aug 30 18:39:16 loganalysisclass1.novalocal systemd[1]: Starting Elasticsearch...
Aug 30 18:40:31 loganalysisclass1.novalocal systemd[1]: elasticsearch.service: start operation timed out. Terminating.
Aug 30 18:40:36 loganalysisclass1.novalocal systemd-entrypoint[2338753]: ERROR: Elasticsearch exited unexpectedly
Aug 30 18:40:36 loganalysisclass1.novalocal systemd[1]: elasticsearch.service: Main process exited, code=exited, status=137/n/a
Aug 30 18:40:36 loganalysisclass1.novalocal systemd[1]: elasticsearch.service: Failed with result 'timeout'.
Aug 30 18:40:36 loganalysisclass1.novalocal systemd[1]: Failed to start Elasticsearch.

Ohh I say / realized this

[2022-08-26T20:28:32,283][ERROR][o.e.b.Elasticsearch      ] [loganalysisclass1.novalocal] fatal exception while booting Elasticsearch
java.nio.file.AccessDeniedException: /var/run/elasticsearch

Exact how did you install elasticsearch using what method and did you change any file permissions

See this thread

This is how my permissions look for

/var/run/elasticsearch

drwxr-xr-x  2 elasticsearch elasticsearch   60 Aug 22 16:54 elasticsearch

so perhaps you need

sudo chown -R elasticsearch:elasticsearch /var/run/elasticsearch

I do not see a file called elasticsearch in /var/run/ directory.
To install elasticsearch, I downloaded the rpm from here and used below cmd.

yum install --enablerepo=elasticsearch elasticsearch

I did not change any file permissions.
Did I miss a step?

Something does not seem right... I am not a yum expert.

I would re-install something is not right.

Please follow the docs here

I suspect you did not run with sudo

sudo yum install --enablerepo=elasticsearch elasticsearch

Also why are you setting this to false, you don't want authentication

# Enable security features
xpack.security.enabled: false

I ran it with sudo.
This is the second time I am installing elasticsearch. I faced the same issue first time as well. Is there some JAVA related environment variable that needs to be set for it to run?

Thanks for your help. There was a issue with the RAM of the virtual machine. Increased the RAM and it works now.

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.