Difference between Elasticsearch Security and Watcher Setting in Elasticsearch

ANUBHAV_GUPTA · August 20, 2023, 6:50am

Hi there,
I am looking to set security on http and transport layer.
But I am confused in what to use between the following:

xpack.security.transport.ssl.verification_mode=certificate
xpack.transport.ssl.verification_mode=certificate

and

xpack.security.http.ssl.verification_mode=certificate
xpack.http.ssl.verification_mode=certificate

How these two are different?
And when am setting xpack.http.ssl.verification_mode=certificate or xpack.security.http.ssl.verification_mode=certificate then also I am getting error regarding alt name in SSL certificate like this:-

curl: (60) SSL: no alternative certificate subject name matches target host name 'ec2-52-53-241-135.us-west-1.compute.amazonaws.com'

stephenb · August 20, 2023, 6:00pm

Hi @ANUBHAV_GUPTA
What version of the Elastic Stack are you using?

ANUBHAV_GUPTA · August 20, 2023, 6:30pm

hey @stephenb ,
I am using 8.7.0.

stephenb · August 20, 2023, 7:19pm

How did you install?

this all should have been taken care of if you just did a normal install.

These pages show the correct settings.

If you are doing all the security manually

Transport Level (interface) Security

HTTP Interface Security

None of this has a direct relationship to watcher other than you need security enabled to use watcher.

Here is the list of TLS setting

the

xpack.security.* setting are the correct form, not sure where you got the others from

ANUBHAV_GUPTA · August 21, 2023, 3:48am

I am using the docker installation.
What does the watcher setting mean in Elasticsearch?

stephenb · August 21, 2023, 3:52am

Nothing unless you are specifically trying to use watcher.

Are you using watcher?

You will need to provide much more information if you are.

ANUBHAV_GUPTA · August 21, 2023, 3:56am

I am actually trying to use my own set of certificate to set up security.
But while setting xpack.security.http.ssl.verification_mode=certificate am still getting the hostname verification.

curl: (60) SSL: no alternative certificate subject name matches target host name 'ec2-52-53-241-135.us-west-1.compute.amazonaws.com'

According to the verification_mode set to certificate, Should'nt it skip the hostname check?

stephenb · August 21, 2023, 4:11am

Proper syntax is

xpack.security.http.ssl.verification_mode: certificate

Not =

So it is probably being ignored

ANUBHAV_GUPTA · August 21, 2023, 4:24am

I am passing this as an env variable in docker run command.
So, I believe the syntax is fine here, Isn't it?

stephenb · August 21, 2023, 4:34am

Perhaps..just guessing because you haven't really clearly shared what you are doing... Just parts...

The settings work when properly configured.

Try none first then proceed from there.

ANUBHAV_GUPTA · August 21, 2023, 4:48am

Sure, here is the command I am using:-

sudo docker run -it --privileged -p 9200:9200 -p 9300:9300 -e discovery.type=multi-node -e "cluster.name=my-elasticsearch-cluster" -e node.name="node-1" -e discovery.seed_hosts=["10.0.0.211","10.0.0.228","192.168.0.136"] -e network.host="0.0.0.0" -e node.roles="master" -e cluster.initial_master_nodes=["node-1","node-2","node-3"] -e network.publish_host="10.0.0.228" -e ELASTIC_PASSWORD=elastic -e xpack.ml.enabled=false -e ES_JAVA_OPTS="-Xms1g -Xmx1g" -v /home/anubhav/certs:/usr/share/elasticsearch/config/certificates -e xpack.security.transport.ssl.key=/usr/share/elasticsearch/config/certificates/key.pem -e xpack.security.transport.ssl.certificate=/usr/share/elasticsearch/config/certificates/cert.pem -e xpack.security.transport.ssl.enabled=true -e xpack.security.transport.ssl.verification_mode=certificate -e xpack.security.transport.ssl.certificate_authorities=/usr/share/elasticsearch/config/certificates/cacert.pem -e xpack.security.http.ssl.enabled=true -e xpack.security.http.ssl.verification_mode=certificate -e xpack.security.http.ssl.certificate_authorities=/usr/share/elasticsearch/config/certificates/cacert.pem -e xpack.security.http.ssl.key=/usr/share/elasticsearch/config/certificates/key.pem -e xpack.security.http.ssl.certificate=/usr/share/elasticsearch/config/certificates/cert.pem elasticsearch:8.7.0

stephenb · August 21, 2023, 5:18am

Ok...so a little more

xpack.security.http.ssl.verification_mode
(Static) The SSL settings in xpack.security.http.ssl control a server context for TLS, which defines the settings for the TLS connection. The use of verification_mode in a TLS server is discouraged. Defines how to verify the certificates presented by another party in the TLS connection:

The key being...

Defines how to verify the certificates presented by another party in the TLS connection:

Sorry I was not clear what you were doing...

You should not mess with that setting.

What you are doing is using curl a client

When you connect with curl ... It is curl that is verifying the cert not elasticsearch...

That is the error you are seeing

For curl... You need to provide the CA or use the -k option

Try

curl -v -k -u elastic https://theelastichost:9200

And / or you can try the --cacert option and provide a path to the CA

ANUBHAV_GUPTA · August 21, 2023, 6:43am

Actually i am trying the same thing from beginning, which you suggested, here is the curl request:-

REQUEST:-
curl --cacert ./ca.pem https://elastic:elastic@ec2-52-53-241-135.us-west-1.compute.amazonaws.com:9200

RESPONSE:-
curl: (60) SSL: no alternative certificate subject name matches target host name 'ec2-52-53-241-135.us-west-1.compute.amazonaws.com'
More details here: curl - SSL CA Certificates
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

stephenb · August 21, 2023, 6:48am

So you have a self signed cert that is appears not to be correct.

If you had run curl like I suggested with the -v option it would have given you a lot more details.

Either your cert is not correct and / or you are not presenting the correct CA or both.

Run with -v you will get more detail, then you should be able to discount... This is normal cert stuff

zqc0512 · August 21, 2023, 7:11am

see the cert file use hostname or domainname
change xpack.security.http.ssl.verification_mode: none
you can use ssl not validate cert

ANUBHAV_GUPTA · August 21, 2023, 7:56am

setting verification_mode=none will hamper the security more.
I am looking for some way, if the hostname verification can be turned off.

stephenb · August 21, 2023, 2:09pm

Hi @ANUBHAV_GUPTA I do not think that will fix the issue anyway.

I think Perhaps you are confused where the error is coming from and what that setting is used for.

The error is coming from curl not elasticsearch.

The setting you are using validates the certificate that is presented to elasticsearch from the client for mutual verification...

curl is not requesting mutual validation. Therefore that setting has no effect.

curl is validating the elasticsearch cert on its side... Not the server / elasticsearch side

Please run the following and show both the commands and full output

curl -v -k ....

curl -v --cacert...

Please show the full command and full output.

ANUBHAV_GUPTA · August 21, 2023, 3:43pm

Hey @stephenb ,
Here is the output for the commands you asked for:-

curl -v --cacert /home/anubhav/Downloads/ca.pem https://elastic:elastic@ec2-13-57-211-186.us-west-1.compute.amazonaws.com:9200

Trying 13.57.211.186:9200...

TCP_NODELAY set

Connected to ec2-13-57-211-186.us-west-1.compute.amazonaws.com (13.57.211.186) port >9200 (#0)

ALPN, offering h2

ALPN, offering http/1.1

successfully set certificate verify locations:

CAfile: /home/anubhav/Downloads/ca.pem
CApath: /etc/ssl/certs

TLSv1.3 (OUT), TLS handshake, Client hello (1):

TLSv1.3 (IN), TLS handshake, Server hello (2):

TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):

TLSv1.3 (IN), TLS handshake, Certificate (11):

TLSv1.3 (IN), TLS handshake, CERT verify (15):

TLSv1.3 (IN), TLS handshake, Finished (20):

TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):

TLSv1.3 (OUT), TLS handshake, Finished (20):

SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384

ALPN, server did not agree to a protocol

Server certificate:

subject: CN=ec2-13-57-211-186.us-west-1.compute.amazonaws.com

start date: Aug 21 15:32:22 2023 GMT

expire date: Nov 19 15:32:22 2023 GMT

subjectAltName: host "ec2-13-57-211-186.us-west-1.compute.amazonaws.com" matched cert's "ec2-13-57-211-186.us-west-1.compute.amazonaws.com"

issuer: 1.3.6.1.4.1.49690.1.3.1=9a7dd16d-149e-45a9-a593-185361eff3b6; 1.3.6.1.4.1.49690.1.3.2=6d8bcb2b-327d-4de3-8479-825b38da3dcc; CN=Default Enclave Zone Root

SSL certificate verify ok.

Server auth using Basic with user 'elastic'
GET / HTTP/1.1
Host: ec2-13-57-211-186.us-west-1.compute.amazonaws.com:9200
Authorization: Basic ZWxhc3RpYzplbGFzdGlj
User-Agent: curl/7.68.0
Accept: /

TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):

Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< X-elastic-product: Elasticsearch
< content-type: application/json
< content-length: 525
<
{
"name" : "node-1",
"cluster_name" : "my-elasticsearch-cluster",
"cluster_uuid" : "na",
"version" : {
"number" : "8.7.0",
"build_flavor" : "default",
"build_type" : "docker",
"build_hash" : "09520b59b6bc1057340b55750186466ea715e30e",
"build_date" : "2023-03-27T16:31:09.816451435Z",
"build_snapshot" : false,
"lucene_version" : "9.5.0",
"minimum_wire_compatibility_version" : "7.17.0",
"minimum_index_compatibility_version" : "7.0.0"
},
"tagline" : "You Know, for Search"
}

Connection #0 to host ec2-13-57-211-186.us-west-1.compute.amazonaws.com left intact

curl -v -k https://elastic:elastic@ec2-13-57-211-186.us-west-1.compute.amazonaws.com:9200

Trying 13.57.211.186:9200...

TCP_NODELAY set

Connected to ec2-13-57-211-186.us-west-1.compute.amazonaws.com (13.57.211.186) port 9200 (#0)

ALPN, offering h2

ALPN, offering http/1.1

successfully set certificate verify locations:

CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs

TLSv1.3 (OUT), TLS handshake, Client hello (1):

TLSv1.3 (IN), TLS handshake, Server hello (2):

TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):

TLSv1.3 (IN), TLS handshake, Certificate (11):

TLSv1.3 (IN), TLS handshake, CERT verify (15):

TLSv1.3 (IN), TLS handshake, Finished (20):

TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):

TLSv1.3 (OUT), TLS handshake, Finished (20):

SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384

ALPN, server did not agree to a protocol

Server certificate:

subject: CN=ec2-13-57-211-186.us-west-1.compute.amazonaws.com

start date: Aug 21 15:32:22 2023 GMT

expire date: Nov 19 15:32:22 2023 GMT

issuer: 1.3.6.1.4.1.49690.1.3.1=9a7dd16d-149e-45a9-a593-185361eff3b6; 1.3.6.1.4.1.49690.1.3.2=6d8bcb2b-327d-4de3-8479-825b38da3dcc; CN=Default Enclave Zone Root

SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.

Server auth using Basic with user 'elastic'
GET / HTTP/1.1
Host: ec2-13-57-211-186.us-west-1.compute.amazonaws.com:9200
Authorization: Basic ZWxhc3RpYzplbGFzdGlj
User-Agent: curl/7.68.0
Accept: /

TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):

Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< X-elastic-product: Elasticsearch
< content-type: application/json
< content-length: 525
<
{
"name" : "node-1",
"cluster_name" : "my-elasticsearch-cluster",
"cluster_uuid" : "na",
"version" : {
"number" : "8.7.0",
"build_flavor" : "default",
"build_type" : "docker",
"build_hash" : "09520b59b6bc1057340b55750186466ea715e30e",
"build_date" : "2023-03-27T16:31:09.816451435Z",
"build_snapshot" : false,
"lucene_version" : "9.5.0",
"minimum_wire_compatibility_version" : "7.17.0",
"minimum_index_compatibility_version" : "7.0.0"
},
"tagline" : "You Know, for Search"
}

Connection #0 to host ec2-13-57-211-186.us-west-1.compute.amazonaws.com left intact

stephenb · August 21, 2023, 3:58pm

@ANUBHAV_GUPTA Cool Thanks!

So we can see the first command:

curl -v --cacert /home/anubhav/Downloads/ca.pem https://elastic:elastic@ec2-13-57-211-186.us-west-1.compute.amazonaws.com:9200

connects successfully and is fully valid when you use the CA Cert. because your elasticsearch cert appears to be a Self-Signed cert ... not a publicly signed cert that is why you need to provide the CA to the client so the client (curl) can do client-side validation

So curl and elasticsearch are operating properly exactly as expected...

So you will need to provide the CA cert for other client connections such as filebeat, logstash etc.

You can / should take out the xpack.security.http.ssl.verification_mode=certificate unless you are very clear on the form of Mutual Validation you want to accomplish if / when you ever use that... you would need to specify on the client side you want to do mutual TLS if you are not familiar with that perhaps you could read up on it.

ANUBHAV_GUPTA · August 21, 2023, 4:35pm

thanks @stephenb,
I would also like to know, How can i connect kibana to this?
What pair of certs do I need to provide in Kibana docker run command?

Topic		Replies	Views
The differences between xpack.security.http.ssl. and xpack.http.ssl. settings Elasticsearch elastic-stack-security	2	736	February 25, 2019
What is the difference between xpack.security.http.ssl.verification_mode and xpack.http.ssl.verification_mode Elasticsearch elastic-stack-security	12	445	April 14, 2023
Watcher Enabling in ES configuration Elasticsearch elastic-stack-alerting	7	430	October 22, 2020
Certificate for <localhost> doesn't match any of the subject alternative names Elasticsearch elastic-stack-alerting	4	8613	July 15, 2019
What is the difference between xpack.security.http.ssl.verification_mode and xpack.http.ssl.verification_mode Elasticsearch	1	198	March 24, 2023

Difference between Elasticsearch Security and Watcher Setting in Elasticsearch

Related topics