How do I host an Elasticsearch server on a local machine and allow other machines to access that data?

I have an Elasticsearch server running on a local machine with an ip address: {local_machine_ip}. I have another computer that has an ip {client_ip}. How do I allow the client computer to access the Elasticsearch server on the local_machine_ip?

I believe you need to configure network.host.

This is how my elasticsearch.yml file looks. Is this correct?

transport.host: localhost
transport.tcp.port: 9300
http.port: 9200
network.host: 0.0.0.0

All you need to set is that one, everything else there is defaults, and transport.host will be over ridden by your network.host.

Ok, and how would I connect to the server (on the other machine) using python?

Connect to the hostname/IP of the server on port 9200.

Would this be the proper syntax?

client = Elasticsearch(
    "https://{public_ip_of_host_machine}:9200",
    basic_auth=("elastic", ELASTIC_PASSWORD)
)

print(client.info())

When I run this, I get this error: elastic_transport.ConnectionTimeout: Connection timed out.

Can you curl https://{public_ip_of_host_machine}:9200 from the host you are running the code on? What does it respond with?

curl: (28) Failed to connect to {public_ip_of_host_machine} port 9200 after 75017 ms: Operation timed out

You might need to post your Elasticsearch logs and config then please.

These are the logs.

dk], using bundled JDK [true]
[2022-06-21T20:37:49,753][INFO ][o.e.n.Node               ] [account] JVM arguments [-Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -Djava.security.manager=allow, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Djava.locale.providers=SPI,COMPAT, --add-opens=java.base/java.io=ALL-UNNAMED, -XX:+UseG1GC, -Djava.io.tmpdir=C:\Users\ACCOUNT~1\AppData\Local\Temp\2\elasticsearch, -XX:+HeapDumpOnOutOfMemoryError, -XX:+ExitOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Xms8191m, -Xmx8191m, -XX:MaxDirectMemorySize=4294967296, -XX:InitiatingHeapOccupancyPercent=30, -XX:G1ReservePercent=25, -Delasticsearch, -Des.path.home=C:\Users\account_admin\Downloads\elasticsearch-8.2.3-windows-x86_64\elasticsearch-8.2.3, -Des.path.conf=C:\Users\account_admin\Downloads\elasticsearch-8.2.3-windows-x86_64\elasticsearch-8.2.3\config, -Des.distribution.flavor=default, -Des.distribution.type=zip, -Des.bundled_jdk=true]
[2022-06-21T20:37:55,968][INFO ][o.e.p.PluginsService     ] [account] loaded module [aggs-matrix-stats]
[2022-06-21T20:37:55,969][INFO ][o.e.p.PluginsService     ] [account] loaded module [analysis-common]
[2022-06-21T20:37:55,969][INFO ][o.e.p.PluginsService     ] [account] loaded module [constant-keyword]
[2022-06-21T20:37:55,969][INFO ][o.e.p.PluginsService     ] [account] loaded module [data-streams]
[2022-06-21T20:37:55,970][INFO ][o.e.p.PluginsService     ] [account] loaded module [frozen-indices]
[2022-06-21T20:37:55,970][INFO ][o.e.p.PluginsService     ] [account] loaded module [ingest-common]
[2022-06-21T20:37:55,971][INFO ][o.e.p.PluginsService     ] [account] loaded module [ingest-geoip]
[2022-06-21T20:37:55,971][INFO ][o.e.p.PluginsService     ] [account] loaded module [ingest-user-agent]
[2022-06-21T20:37:55,972][INFO ][o.e.p.PluginsService     ] [account] loaded module [kibana]
[2022-06-21T20:37:55,972][INFO ][o.e.p.PluginsService     ] [account] loaded module [lang-expression]
[2022-06-21T20:37:55,972][INFO ][o.e.p.PluginsService     ] [account] loaded module [lang-mustache]
[2022-06-21T20:37:55,973][INFO ][o.e.p.PluginsService     ] [account] loaded module [lang-painless]
[2022-06-21T20:37:55,973][INFO ][o.e.p.PluginsService     ] [account] loaded module [legacy-geo]
[2022-06-21T20:37:55,974][INFO ][o.e.p.PluginsService     ] [account] loaded module [mapper-extras]
[2022-06-21T20:37:55,974][INFO ][o.e.p.PluginsService     ] [account] loaded module [mapper-version]
[2022-06-21T20:37:55,975][INFO ][o.e.p.PluginsService     ] [account] loaded module [parent-join]
[2022-06-21T20:37:55,975][INFO ][o.e.p.PluginsService     ] [account] loaded module [percolator]
[2022-06-21T20:37:55,975][INFO ][o.e.p.PluginsService     ] [account] loaded module [rank-eval]
[2022-06-21T20:37:55,976][INFO ][o.e.p.PluginsService     ] [account] loaded module [reindex]
[2022-06-21T20:37:55,976][INFO ][o.e.p.PluginsService     ] [account] loaded module [repositories-metering-api]
[2022-06-21T20:37:55,977][INFO ][o.e.p.PluginsService     ] [account] loaded module [repository-azure]
[2022-06-21T20:37:55,980][INFO ][o.e.p.PluginsService     ] [account] loaded module [repository-encrypted]
[2022-06-21T20:37:55,980][INFO ][o.e.p.PluginsService     ] [account] loaded module [repository-gcs]
[2022-06-21T20:37:55,981][INFO ][o.e.p.PluginsService     ] [account] loaded module [repository-s3]
[2022-06-21T20:37:55,981][INFO ][o.e.p.PluginsService     ] [account] loaded module [repository-url]
[2022-06-21T20:37:55,981][INFO ][o.e.p.PluginsService     ] [account] loaded module [runtime-fields-common]
[2022-06-21T20:37:55,982][INFO ][o.e.p.PluginsService     ] [account] loaded module [search-business-rules]
[2022-06-21T20:37:55,982][INFO ][o.e.p.PluginsService     ] [account] loaded module [searchable-snapshots]
[2022-06-21T20:37:55,983][INFO ][o.e.p.PluginsService     ] [account] loaded module [snapshot-based-recoveries]
[2022-06-21T20:37:55,983][INFO ][o.e.p.PluginsService     ] [account] loaded module [snapshot-repo-test-kit]
[2022-06-21T20:37:55,983][INFO ][o.e.p.PluginsService     ] [account] loaded module [spatial]
[2022-06-21T20:37:55,984][INFO ][o.e.p.PluginsService     ] [account] loaded module [transform]
[2022-06-21T20:37:55,984][INFO ][o.e.p.PluginsService     ] [account] loaded module [transport-netty4]
[2022-06-21T20:37:55,985][INFO ][o.e.p.PluginsService     ] [account] loaded module [unsigned-long]
[2022-06-21T20:37:55,985][INFO ][o.e.p.PluginsService     ] [account] loaded module [vector-tile]
[2022-06-21T20:37:55,985][INFO ][o.e.p.PluginsService     ] [account] loaded module [vectors]
[2022-06-21T20:37:55,986][INFO ][o.e.p.PluginsService     ] [account] loaded module [wildcard]
[2022-06-21T20:37:55,986][INFO ][o.e.p.PluginsService     ] [account] loaded module [x-pack-aggregate-metric]
[2022-06-21T20:37:55,987][INFO ][o.e.p.PluginsService     ] [account] loaded module [x-pack-analytics]
[2022-06-21T20:37:55,988][INFO ][o.e.p.PluginsService     ] [account] loaded module [x-pack-async]
[2022-06-21T20:37:55,993][INFO ][o.e.p.PluginsService     ] [account] loaded module [x-pack-async-search]
[2022-06-21T20:37:55,996][INFO ][o.e.p.PluginsService     ] [account] loaded module [x-pack-autoscaling]
[2022-06-21T20:37:55,996][INFO ][o.e.p.PluginsService     ] [account] loaded module [x-pack-ccr]
[2022-06-21T20:37:55,997][INFO ][o.e.p.PluginsService     ] [account] loaded module [x-pack-core]
[2022-06-21T20:37:55,998][INFO ][o.e.p.PluginsService     ] [account] loaded module [x-pack-deprecation]
[2022-06-21T20:37:55,999][INFO ][o.e.p.PluginsService     ] [account] loaded module [x-pack-enrich]
[2022-06-21T20:37:56,000][INFO ][o.e.p.PluginsService     ] [account] loaded module [x-pack-eql]
[2022-06-21T20:37:56,000][INFO ][o.e.p.PluginsService     ] [account] loaded module [x-pack-fleet]
[2022-06-21T20:37:56,001][INFO ][o.e.p.PluginsService     ] [account] loaded module [x-pack-graph]
[2022-06-21T20:37:56,004][INFO ][o.e.p.PluginsService     ] [account] loaded module [x-pack-identity-provider]
[2022-06-21T20:37:56,006][INFO ][o.e.p.PluginsService     ] [account] loaded module [x-pack-ilm]
[2022-06-21T20:37:56,007][INFO ][o.e.p.PluginsService     ] [account] loaded module [x-pack-logstash]
[2022-06-21T20:37:56,007][INFO ][o.e.p.PluginsService     ] [account] loaded module [x-pack-ml]
[2022-06-21T20:37:56,008][INFO ][o.e.p.PluginsService     ] [account] loaded module [x-pack-monitoring]
[2022-06-21T20:37:56,008][INFO ][o.e.p.PluginsService     ] [account] loaded module [x-pack-ql]
[2022-06-21T20:37:56,009][INFO ][o.e.p.PluginsService     ] [account] loaded module [x-pack-rollup]
[2022-06-21T20:37:56,009][INFO ][o.e.p.PluginsService     ] [account] loaded module [x-pack-security]
[2022-06-21T20:37:56,010][INFO ][o.e.p.PluginsService     ] [account] loaded module [x-pack-shutdown]
[2022-06-21T20:37:56,011][INFO ][o.e.p.PluginsService     ] [account] loaded module [x-pack-sql]
[2022-06-21T20:37:56,012][INFO ][o.e.p.PluginsService     ] [account] loaded module [x-pack-stack]
[2022-06-21T20:37:56,013][INFO ][o.e.p.PluginsService     ] [account] loaded module [x-pack-text-structure]
[2022-06-21T20:37:56,018][INFO ][o.e.p.PluginsService     ] [account] loaded module [x-pack-voting-only-node]
[2022-06-21T20:37:56,019][INFO ][o.e.p.PluginsService     ] [account] loaded module [x-pack-watcher]
[2022-06-21T20:37:56,020][INFO ][o.e.p.PluginsService     ] [account] no plugins loaded
[2022-06-21T20:37:56,422][INFO ][o.e.e.NodeEnvironment    ] [account] using [1] data paths, mounts [[Windows (C:)]], net usable_space [105.9gb], net total_space [126.4gb], types [NTFS]
[2022-06-21T20:37:56,423][INFO ][o.e.e.NodeEnvironment    ] [account] heap size [8gb], compressed ordinary object pointers [true]
[2022-06-21T20:37:56,694][INFO ][o.e.n.Node               ] [account] node name [account], node ID [ig-irOulTYqfXsohxaxZuA], cluster name [elasticsearch], roles [data_cold, data, remote_cluster_client, master, data_warm, data_content, transform, data_hot, ml, data_frozen, ingest]
[2022-06-21T20:38:04,828][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [account] [controller/400] [Main.cc@123] controller (64 bit): Version 8.2.3 (Build 537f37a54d22f1) Copyright (c) 2022 Elasticsearch BV
[2022-06-21T20:38:05,148][INFO ][o.e.x.s.Security         ] [account] Security is enabled
[2022-06-21T20:38:05,664][INFO ][o.e.x.s.a.Realms         ] [account] license mode is [trial], currently licensed security realms are [reserved/reserved,file/default_file,native/default_native]
[2022-06-21T20:38:05,683][INFO ][o.e.x.s.a.s.FileRolesStore] [account] parsed [0] roles from file [C:\Users\account_admin\Downloads\elasticsearch-8.2.3-windows-x86_64\elasticsearch-8.2.3\config\roles.yml]

[2022-06-21T20:38:07,699][INFO ][o.e.t.n.NettyAllocator   ] [account] creating NettyAllocator with the following configs: [name=elasticsearch_configured, chunk_size=1mb, suggested_max_allocation_size=1mb, factors={es.unsafe.use_netty_default_chunk_and_page_size=false, g1gc_enabled=true, g1gc_region_size=4mb}]
[2022-06-21T20:38:07,740][INFO ][o.e.i.r.RecoverySettings ] [account] using rate limit [40mb] with [default=40mb, read=0b, write=0b, max=0b]
[2022-06-21T20:38:07,792][INFO ][o.e.d.DiscoveryModule    ] [account] using discovery type [multi-node] and seed hosts providers [settings]
[2022-06-21T20:38:10,150][INFO ][o.e.n.Node               ] [account] initialized
[2022-06-21T20:38:10,150][INFO ][o.e.n.Node               ] [account] starting ...
[2022-06-21T20:38:10,198][INFO ][o.e.x.s.c.f.PersistentCache] [account] persistent cache index loaded
[2022-06-21T20:38:10,200][INFO ][o.e.x.d.l.DeprecationIndexingComponent] [account] deprecation component started
[2022-06-21T20:38:10,390][INFO ][o.e.t.TransportService   ] [account] publish_address {127.0.0.1:9300}, bound_addresses {127.0.0.1:9300}, {[::1]:9300}
[2022-06-21T20:38:11,068][INFO ][o.e.c.c.Coordinator      ] [account] cluster UUID [i39FIYZKQo-BXpacaqbCVA]
[2022-06-21T20:38:11,243][INFO ][o.e.c.s.MasterService    ] [account] elected-as-master ([1] nodes joined)[_FINISH_ELECTION_, {account}{ig-irOulTYqfXsohxaxZuA}{wdsGiGk3ShG64Y90aiqZQA}{127.0.0.1}{127.0.0.1:9300}{cdfhilmrstw} completing election], term: 16, version: 285, delta: master node changed {previous [], current [{account}{ig-irOulTYqfXsohxaxZuA}{wdsGiGk3ShG64Y90aiqZQA}{127.0.0.1}{127.0.0.1:9300}{cdfhilmrstw}]}
[2022-06-21T20:38:11,469][INFO ][o.e.c.s.ClusterApplierService] [account] master node changed {previous [], current [{account}{ig-irOulTYqfXsohxaxZuA}{wdsGiGk3ShG64Y90aiqZQA}{127.0.0.1}{127.0.0.1:9300}{cdfhilmrstw}]}, term: 16, version: 285, reason: Publication{term=16, version=285}
[2022-06-21T20:38:11,697][INFO ][o.e.h.AbstractHttpServerTransport] [account] publish_address {10.0.0.4:9200}, bound_addresses {[::]:9200}
[2022-06-21T20:38:11,717][INFO ][o.e.n.Node               ] [account] started
[2022-06-21T20:38:12,301][INFO ][o.e.l.LicenseService     ] [account] license [10e1f812-3c45-4680-923b-6bd23e151643] mode [basic] - valid
[2022-06-21T20:38:12,309][INFO ][o.e.x.s.a.Realms         ] [account] license mode is [basic], currently licensed security realms are [reserved/reserved,file/default_file,native/default_native]
[2022-06-21T20:38:12,317][INFO ][o.e.g.GatewayService     ] [account] recovered [11] indices into cluster_state
[2022-06-21T20:38:12,562][ERROR][o.e.i.g.GeoIpDownloader  ] [account] exception during geoip databases update
org.elasticsearch.ElasticsearchException: not all primary shards of [.geoip_databases] index are active
        at org.elasticsearch.ingest.geoip.GeoIpDownloader.updateDatabases(GeoIpDownloader.java:135) ~[ingest-geoip-8.2.3.jar:8.2.3]
        at org.elasticsearch.ingest.geoip.GeoIpDownloader.runDownloader(GeoIpDownloader.java:275) [ingest-geoip-8.2.3.jar:8.2.3]
        at org.elasticsearch.ingest.geoip.GeoIpDownloaderTaskExecutor.nodeOperation(GeoIpDownloaderTaskExecutor.java:102) [ingest-geoip-8.2.3.jar:8.2.3]
        at org.elasticsearch.ingest.geoip.GeoIpDownloaderTaskExecutor.nodeOperation(GeoIpDownloaderTaskExecutor.java:48) [ingest-geoip-8.2.3.jar:8.2.3]
        at org.elasticsearch.persistent.NodePersistentTasksExecutor$1.doRun(NodePersistentTasksExecutor.java:42) [elasticsearch-8.2.3.jar:8.2.3]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:773) [elasticsearch-8.2.3.jar:8.2.3]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) [elasticsearch-8.2.3.jar:8.2.3]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
        at java.lang.Thread.run(Thread.java:833) [?:?]
[2022-06-21T20:38:14,694][INFO ][o.e.c.r.a.AllocationService] [account] current.health="GREEN" message="Cluster health status changed from [RED] to [GREEN] (reason: [shards started [[.ds-ilm-history-5-2022.06.21-000001][0], [.ds-.logs-deprecation.elasticsearch-default-2022.06.21-000001][0]]])." previous.health="RED" reason="shards started [[.ds-ilm-history-5-2022.06.21-000001][0], [.ds-.logs-deprecation.elasticsearch-default-2022.06.21-000001][0]]"

You can see here that it's bound to 10.0.0.4:9200, so unless you are on the same network, or have access to that network, then you won't be able to connect to it.

This is my config (elasticsearch.yml) file:




# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
#node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
#path.data: /path/to/data
#
# Path to log files:
#
#path.logs: /path/to/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# By default Elasticsearch is only accessible on localhost. Set a different
# address here to expose this node on the network:
#
#network.host: {cut_out}
#
# By default Elasticsearch listens for HTTP traffic on the first free port it
# finds starting at 9200. Set a specific HTTP port here:
#
#http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
#cluster.initial_master_nodes: ["node-1", "node-2"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# --------------------------------- Readiness ----------------------------------
#
# Enable an unauthenticated TCP readiness endpoint on localhost
#
#readiness.port: 9399
#
# ---------------------------------- Various -----------------------------------
#
# Allow wildcard deletion of indices:
#
#action.destructive_requires_name: false

#----------------------- BEGIN SECURITY AUTO CONFIGURATION -----------------------
#
# The following settings, TLS certificates, and keys have been automatically      
# generated to configure Elasticsearch security features on 20-06-2022 19:26:35
#
# --------------------------------------------------------------------------------

# Enable security features
xpack.security.enabled: true

xpack.security.enrollment.enabled: true

# Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents
xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12

# Enable encryption and mutual authentication between cluster nodes
xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12
# Create a new cluster with the current node only
# Additional nodes can still join the cluster later
cluster.initial_master_nodes: ["account"]

# Allow HTTP API connections from anywhere
# Connections are encrypted and require user authentication
http.host: 0.0.0.0


network.host: {public_ip_of_host_machine}

# Allow other nodes to join the cluster from anywhere
# Connections are encrypted and mutually authenticated
#transport.host: 0.0.0.0

#----------------------- END SECURITY AUTO CONFIGURATION -------------------------

What is the solution to this? Sorry, if it is not seeming obvious to me. I'm not completely familiar/comfortable with network/https/http.

Does this host actually have an interface with a public IP attached to it? Because this doesn't match your logs from above.

The host is a virtual machine and has an IP that is needed to connect to it.

It might be worth chatting to one of your network/system administrators internally.

Elasticsearch is listening on the IP that you requested - 10.0.0.4 - and you need to be able to access that from your own host. If you cannot do that, then it's a network issue.

Ok, thanks for your help. I'll look into that. By the way, is there an example of this being done out there that I could use as a template? Just asking if there is a way I can get a better understanding of how to do this.

Generally you would configure Elasticsearch to listen to network.host: 0.0.0.0, so that it'll attach to any available interface/IP on the host.

Other than that, anything else really depends on your network, which is outside our control.