How do you bind Elasticsearch server to public(ipV4) address?

[DataStorageMuse]

I'm trying to use a http_ca certificate to connect to an Elasticsearch server (using Python), and I was wondering how I can bind an instance to a public IP address. When I try to connect, I get this error:

elastic_transport.TlsError: TLS error caused by: TlsError(TLS error caused by: SSLError(hostname '{public ipv4_address}' doesn't match either of 'localhost', '127.0.0.1',{private_ip}, {cluster_name}))

[warkolm (Mark Walkom)]

Please share your elasticsearch.yml.

[DataStorageMuse]

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
#node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
#path.data: /path/to/data
#
# Path to log files:
#
#path.logs: /path/to/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# By default Elasticsearch is only accessible on localhost. Set a different
# address here to expose this node on the network:
#
#network.host: {private_ip}
#
# By default Elasticsearch listens for HTTP traffic on the first free port it
# finds starting at 9200. Set a specific HTTP port here:
#
#http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
#cluster.initial_master_nodes: ["node-1", "node-2"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# --------------------------------- Readiness ----------------------------------
#
# Enable an unauthenticated TCP readiness endpoint on localhost
#
#readiness.port: 9399
#
# ---------------------------------- Various -----------------------------------
#
# Allow wildcard deletion of indices:
#
#action.destructive_requires_name: false

#----------------------- BEGIN SECURITY AUTO CONFIGURATION -----------------------
#
# The following settings, TLS certificates, and keys have been automatically      
# generated to configure Elasticsearch security features on 23-06-2022 15:36:16
#
# --------------------------------------------------------------------------------

# Enable security features
xpack.security.enabled: true

xpack.security.enrollment.enabled: true

# Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents
xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12

# Enable encryption and mutual authentication between cluster nodes
xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12
# Create a new cluster with the current node only
# Additional nodes can still join the cluster later
cluster.initial_master_nodes: [{some_name}]

# Allow HTTP API connections from anywhere
# Connections are encrypted and require user authentication
# http.host: 0.0.0.0

network.host: 0.0.0.0



# Allow other nodes to join the cluster from anywhere
# Connections are encrypted and mutually authenticated
#transport.host: 0.0.0.0

#----------------------- END SECURITY AUTO CONFIGURATION -------------------------

[DataStorageMuse]

By the way, I can connect when I use the certification fingerprint, but since I'm running this on a system that only currently supports Python 3.9-, I need to use a CA certificate.

[warkolm (Mark Walkom)]

@DataStorageMuse

#network.host: {private_ip}

That needs to be uncommented and set to 0.0.0.0, which means it'll use any address the host has.

[DataStorageMuse]

I added the variable here: network.host: 0.0.0.0, no?

Please scroll to the bottom.

[DataStorageMuse]

client = Elasticsearch(
    "https://{ipv4_address}:9200",
    ca_certs="http_ca.crt", 
    basic_auth=("elastic", ELASTIC_PASSWORD)
)

Here is my Python code if that helps.

[warkolm (Mark Walkom)]

Ah right, most configs I have seen just uncomment that section rather than adding it additionally at the bottom as it minimises confusion like this

If that's been set then you should see a line in the Elasticsearch logs that mentions binding, that will let you know what interfaces it is using.

[DataStorageMuse]

I see this:

[2022-06-28T21:19:35,746][INFO ][o.e.t.TransportService   ] [{some_name}] publish_address {{private_IP}:9300}, bound_addresses {[::]:9300}

Is there a way to get it to bound to the Public IP instead?

[leandrojmp (Leandro Pereira)]

Where are you running your instance?

You can only bind a service to an IP address that is available in the server and your server probably do not have an IP address available in one of its network interfaces.

You need to bind it to 0.0.0.0 and check in your network infrastructure how an external request will arrive at your server.

For example, in a cloud service like AWS, when you attach an Elastic IP to an instance, the public IP is not available in the instance, but AWS will redirect any request to the public IP to the private IP of your instance.

[DataStorageMuse]

Ok, but when I use a certification fingerprint, I can connect, implying that the infrastructure can redirect a request from a public IP to private IP. However, they problem is the http_ca certificate file doesn't allow me to use the public IP address. I was wondering how I could use the http_ca certificate as authentication, allowing me to use the public IP.

[stephenb (Stephen Brown)]

@DataStorageMuse

However, they problem is the http_ca certificate file doesn't allow me to use the public IP address.

Ok @DataStorageMuse follow me on this (I am not the cert expert ... but I can get around)

So The problem is most likely that that the http.p12 cert does not contain the external IP address that you are using... in fact I am sure it does not

xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12

you can check with...

openssl s_client -connect <yourexternalip>:9200 < /dev/null | openssl x509 -noout -text | grep -C3 -i dns

and what you are going to get is... something like

epth=1 CN = Elasticsearch security auto-configuration HTTP CA
verify error:num=19:self signed certificate in certificate chain
verify return:0
poll error                keyid:D7:2D:86:BF:E7:C5:B6:20:67:20:40:5D:8F:AF:B3:85:2E:DF:B1:FC

            X509v3 Subject Alternative Name: 
                IP Address:FE80:0:0:0:4001:AFF:FEA8:C, DNS:localhost, IP Address:0:0:0:0:0:0:0:1, IP Address:127.0.0.1, IP Address:10.168.0.12, DNS:stephenb-es-8-test
            X509v3 Basic Constraints: 
                CA:FALSE
    Signature Algorithm: sha256WithRSAEncryption

Note Subject Alternative Name... name all the IP addresses they are all the local ones... not that public IP... and the hostname ... That is why the cert fails externally... period ... will never work via the public IP unless you use --insecure or verification_mode : none

Now you ask... that's all great Mr @stephenb but how do I fix it ....

Well you would need to generate a cert that DOES include that external address...

You could create we real cert through lets encrypt etc with real DNS

Or you can use the elasticsearch self signing tools

Take a look at this very detailed post on how to do this manually ... or you can piece it together with this and this

THE KEY IS when you get to the part that says

Enter all the IP addresses that you need, one per line.
When you are done, press <ENTER> once more to move on to the next step.

10.168.0.116
127.0.0.1

Make sure you add that external address... as well

You will to then make sure you use the new CA in all the right places and the correct certs...

Ohhh BTW A complete cheat is in your /etc/host set the hostname to that public ip and it will work too Total Hack but a good check..

Then this worked for me

curl -v --cacert ./http_ca.crt -u elastic:password https://stephenb-es-8-test:9200

[DataStorageMuse]

Thank you @stephenb. This was very useful. Exactly what I was looking for.