cURL POST to Elasticsearch using AD credentials

Elasticsearch and Kibana version 7.4.1 (same machine)
Build type RPM

Hello! Just some background - I have gotten Active Directory authentication working for Kibana, I can log in using AD username/password. I have also set up PKI authentication between Elasticsearch and Kibana and it works. Elasticsearch is also operating in FIPS 140-2 mode successfully.

I am having an issue where I cannot send a cURL POST command to Elasticsearch using AD credentials via the terminal. I am able, however, to log into Kibana using an AD user and use the developer tools section to create a POST command and it works just fine. I am also able to use a cURL POST command from the terminal and specify the local server's PKI certificate & key and this works just fine.

The cURL command I am using is:

curl -u POST ":9200/es_test_1/messages" -H 'Content-Type: application/json' -d'{ "text": "Testing 1234"}

I enter my password and I get this error: SSL certificate problem: unable to get local issuer certificate

If I change the command to include the --cacert option, I get this error: OpenSSL SSL_read: error:14094412:SSL routines:ssl3_read_bytes:sslv3 alert bad certificate, errno 0

I'm not sure why cURL is attempting to use PKI authentication here. I have added the --ntlm option on a cURL different try and got the same bad certificate error. I also have tried --ntlm-proxy, --proxy-user, & --proxy options while specifying my Windows domain controller. I can see the Elasticsearch is connecting to the domain controller by using netstat on the domain controller. However, nothing happens after I put in my domain password.

  • Trying (IP of DC):636...
  • TCP_NODELAY set
  • Connected to (FQDN of DC) (IP) port 636 (#0)
  • allocate connect buffer!
  • Establish HTTP proxy tunnel to (elasticsearch):9200
  • Proxy auth using NTLM with user '(domainUser)'
  • CONNECT (elasticsearch):9200 HTTP/1.1
  • Host: (elasticsearch):9200
  • Proxy-Authorization: NTLM TlRMTVNTUAABAAAABoIIAAAAAAAAAAAAAAAAAAAAAAA=
  • User-Agent: curl/7.66.0
  • Proxy-Connection: Keep-Alive

And that's all that happens. It never finishes the HTTP request.

Does anyone have any insight into this? Seen this before?

Please let me know if more information is needed.

Thanks!

  • Tyler

Slightly unrelated to this topic, and mostly out of curiosity , which Crypto Provider do you use for your FIPS 140 setup ?

It would become obvious if you share your configuration, but it sounds like you have xpack.security.http.ssl.client_authentication: required instead of xpack.security.http.ssl.client_authentication: optional in your configuration

If this is not the case, please share your configuration with us and relevant parts from the elasticsearch.log when you attempt to authenticate with cURL.

You shouldn't need any of these. Elasticsearch will take care of validating your users credentials against your domain controller.

For the password_hashing.algorithm, I am using pbkdf2_10000 (if that is what you're asking for, if not let me know and I'll find the right thing)

Yes! I had http.ssl.client_authentication set to required. I changed it to optional and things worked. Thanks for the assist!! (I think I got confused as to what that xpack setting meant)

And, yes. I didn't need the ntlm or ntlm-proxy options with the cURL request.

Thanks for the help! I really appreciate it!

I was asking mostly how did you set up your Java installation to be in FIPS 140 approved mode ( i.e. which security provider you are using ). As we state in the docs elasticsearch can run in a JVM configured in FIPS 140 approved mode, which presupposes that your JVM is configured in a FIPS 140 approved mode

Oh! Hmm... well, maybe I don't have FIPS 140 mode working then. I just followed the Elasticsearch docs for enabling it within Elasticsearch, which did complain about my p12 certificates initially until I changed them to PEM format. I should have read the documents a little more closely...

I'll work on getting the JVM set to FIPS 140 mode. Thanks for the info, I would have totally missed that. :smiley:

No worries, we plan on revamping our FIPS 140-2 documentation soon. The fact that this was not obvious is useful feedback :slight_smile:

So, I'm at a bit of a loss on how to enabled FIPS mode in the JVM. There doesn't seem to be definitive documentation from Oracle on how to do it for a generic JVM. I found a few of the Github posts relating to FIPS and Elasticsearch, and I know those are mostly for bugs and issues. And, I searched through the forums here and there are very few posts relating to FIPS in any capacity.

I found the java.security file that relates to Elasticsearch on my machine, but I don't really know what to do with it. Do you all have any tips/documentation or Google search keywords that may help me?

Thanks!

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.