Accessing API Without User Credentials

Elastic Stack Elasticsearch
#1

I am trying to build a service monitor and am at the point where I would be making a call to change an index setting in Elasticsearch. Elasticsearch 7.10.0 running and secured with HTTPS. I have the following running:

$ESAPI = @{
  Key = "stuff-Glg"
  Id = "otherstuff-"
  
}
$ESAPI.Cred = [Convert]::ToBase64String([System.Text.Encoding]::Unicode.GetBytes("$($ESAPI.Id):$($ESAPI.Key)"))
$body = '{"blocks.write":false}'
		$headers = @{ 'Authorization' = "ApiKey $($ESApi.Cred)"}
Invoke-RestMethod -Method PUT -Uri "$($ESPro)://$($ESHost):$($ESPort)/$($matches.index)/_settings?pretty" -Body $body -ContentType Application/Json -Headers $headers

I get the below error:

Invoke-RestMethod : {
  "error" : {
    "root_cause" : [
      {
        "type" : "security_exception",
        "reason" : "missing authentication credentials for REST request [/contoso-2020.52/_settings?pretty]",
        "header" : {
          "WWW-Authenticate" : [
            "Basic realm=\"security\" charset=\"UTF-8\"",
            "Bearer realm=\"security\"",
            "ApiKey"
          ]
        }
      }
    ]

Research appears to show that I have to supply the username/password associated with the API key (and testing confirmed this). Is there a way to change index settings without hardcoding user credentials into a script?

#2

Whether you hardcode the credentials or not is up to you - your script could source its credentials from a variety of places, including something like Vault, or a simple configuration file.

Ultimately, if you want to have a protected cluster then every client that connects to the cluster needs to authenticate itself in some way. It is possible to turn off security, or to enable anonymous access, but granting anonymous user the ability to change index settings is pretty dangerous.

An alternative is to use PKI authentication instead, but in effect you're just using a file-based private-key credential instead of a password, so it's not substantially different than having a config file.

#3

I agree, giving an anon user any kind of rights like that is asking for a world of hurt.

I guess I don't see the use of an API key. You need the api key, key id, username, and password...seems like user credentials are redundant. The way Ive seen API keys implemented is they're the credentials to access a system, not a part of the credentials.

#4

Sorry, I completely misread your original post and missed that you were trying to use an API key. That should work - you do not need to provide a username & password alongside an API key.

There's not enough information in your post to be able to tell why your API Key isn't being authenticated. The error message is misleading - if you provide an API key that fails, you will get a missing authentication credentials message, even though you did provide a form of credentials.

One of these is a possible cause:

  1. You're not actually sending the Authorization header
  2. The Authorization header you are sending is in the wrong form
  3. Your API key/id parameters are incorrect, or for a different cluster.
  4. Your API key has expired or been invalidated

I would start with an absolute minimum example, and send the equivalent of:

curl -H "Authorization: ApiKey $(printf "%s:%s" $api_id:$api_key | base64)" \
  "https://${host}:${port}/_security/_authenticate"

And then see what happens. You may find some pointers in the ES node logs.

Sorry, my powershell (I think your script is powershell) isn't good enough to give you an equivalent example.

#5

Turns out the issue is with the base64 conversion. This:

[Convert]::ToBase64String([System.Text.Encoding]::Unicode.GetBytes("$($ESAPI.Id):$($ESAPI.Key)"))

Creates an incorrect base64 encoded string. The correct function is...

[Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $ESAPI.Id, $ESAPI.Key)))

This is .NET I believe, it's not straight PowerShell...so I'm guessing Elasticsearch is expecting base64 encoding of ASCII characters, not Unicode characters...though I've exceeded my knowledge by this point.

#6

Glad to hear you sorted it out.

I'm not an .NET expert, but your explanation makes sense.
To parse the header, we decode the base64 string into bytes and then construct a string by treating those bytes as a UTF8 encoded string.
If they are 7 bit chars (latin alphabet, etc) then ASCII and UTF8 are the same.

I assume .NET's Unicode encoding is UTF16 or UTF32 rather than UTF8.

#7

I would recommend using [Text.Encoding]::UTF8.GetBytes().

There's a PowerShell module, Elastic.Console, that you may be of interest to you.