Trouble when upgrading ES from 7.3.2 to 7.9.3

Elastic Stack Elasticsearch
1

I'm trying to upgrade an ES cluster from 7.3.2 to 7.9.3. I'm configuring ES using Ansible and make REST API calls using Ansible's get_url module. After installation/upgrade and start of ES, it tries to make REST API calls to set user passwords.

So I'm now stuck without knowing what's actually happening. I've upgraded two coordinators and one data/ingest/master node. So I don't dare continuing now since I'm unable to find any information about any changes regarding this very subject. After a few hours of looking in the documentation and searching on Google, I've come to complete stop.

This is what the configuration regarding security looks like:

xpack:
  security:
    enabled: true

Ansible uses basic auth and gets 401 when trying to change password for "elastic":

   "error": {
            "header": {
                "WWW-Authenticate": [
                    "Basic realm=\"security\" charset=\"UTF-8\"", 
                    "Bearer realm=\"security\"", 
                    "ApiKey"
                ]
            }, 
            "reason": "missing authentication credentials for REST request [/_security/user/elastic/_password]", 
            "root_cause": [
                {
                    "header": {
                        "WWW-Authenticate": [
                            "Basic realm=\"security\" charset=\"UTF-8\"", 
                            "Bearer realm=\"security\"", 
                            "ApiKey"
                        ]
                    }, 
                    "reason": "missing authentication credentials for REST request [/_security/user/elastic/_password]", 
                    "type": "security_exception"
                }
            ], 
            "type": "security_exception"
        }

I really need some clarity, please help.
Thanks in advance.

2

There shouldn't be a need to change the password on an upgrade, can you remove that from the process?

That said, can you get the user details that ansible is using and try them?

3

I've tested Ansible's user and it works just fine with curl. I can list nodes as well as post the JSON to the URL mentioned. I get 200 in return as well as an empty body. Ansible is still denied with a 401.

4

I dug a bit deeper and the Ansible module uri has an option called force_basic_auth which makes it all start working again. It doesn't try basic auth on the first try so it seems like there is a bug in Ansible where it can't handle the 401 error that's returned which is otherwise a signal that tells it to use basic auth and try again. So with force_basic_auth, it tells the module to use basic auth without trying without basic auth at first. This is an odd implementation. But I guess there are odd REST APIs out there as well.

My guess is that something changed in the output and Ansible couldn't handle it properly. I'm running a tad older version of Ansible as well so this might not be an issue in newer versions.

This is what the Ansible documentation says about force_basic_auth:

Force the sending of the Basic authentication header upon initial request.
The library used by the uri module only sends authentication information when a webservice responds to an initial request with a 401 status. Since some basic auth services do not properly send a 401, logins will fail.

#solved

Thanks for the help. :smiley:

1 Like
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.