AWS S3 repo configuration

[dj_kill (Kirill)]

Hello.

I'm using Elasticsearch (ECK) Operator 2.7 in OpenShift environment.
All components are on 8.6.2 version.

Configuration for the S3 is pretty simple:

spec:
  version: 8.6.2
  secureSettings:
    - secretName: credentials
      entries:
        - key: AWS_SECRET_ACCESS_KEY
          path: s3.client.default.access_key
        - key: AWS_ACCESS_KEY_ID
          path: s3.client.default.secret_key

and

  nodeSets:
      config:
        s3.client.default.endpoint: "https://s3.eu-west-1.amazonaws.com"
        s3.client.default.region: "eu-west-1"

I tried with region and endpoint and without.
IAM user with provided key has policy attached as it is described here..

But if I try to verify repository I've got AuthorizationHeaderMalformed error.

{
  "name": "ResponseError",
  "meta": {
    "body": {
      "error": {
        "root_cause": [
          {
            "type": "repository_verification_exception",
            "reason": "[platform-backups] path  is not accessible on master node"
          }
        ],
        "type": "repository_verification_exception",
        "reason": "[platform-backups] path  is not accessible on master node",
        "caused_by": {
          "type": "i_o_exception",
          "reason": "Unable to upload object [tests-Sbnv9P08StSsQ9EaD59_uQ/master.dat] using a single upload",
          "caused_by": {
            "type": "amazon_s3_exception",
            "reason": "amazon_s3_exception: /YYYYMMDD/REGION/SERVICE/aws4_request\". (Service: Amazon S3; Status Code: 400; Error Code: AuthorizationHeaderMalformed; Request ID: ....; S3 Extended Request ID: ....; Proxy: null)"
          }
        }
      },
      "status": 500
    },
    "statusCode": 500,
    "headers": {
      "x-opaque-id": "998ea841-e17e-46fe-b9bc-2b6839485374;kibana:application:management:",
      "x-elastic-product": "Elasticsearch",
      "content-type": "application/json;charset=utf-8",
      "content-length": "743"
    },
    "meta": {
      "context": null,
      "request": {
        "params": {
          "method": "POST",
          "path": "/_snapshot/platform-backups/_verify",
          "querystring": "",
          "headers": {
            "user-agent": "Kibana/8.6.2",
            "x-elastic-product-origin": "kibana",
            "authorization": "Basic ....",
            "x-opaque-id": "998ea841-e17e-46fe-b9bc-2b6839485374;kibana:application:management:",
            "x-elastic-client-meta": "es=8.4.0p,js=16.18.1,t=8.2.0,hc=16.18.1",
            "accept": "application/vnd.elasticsearch+json; compatible-with=8,text/plain"
          }
        },
        "options": {
          "opaqueId": "998ea841-e17e-46fe-b9bc-2b6839485374;kibana:application:management:",
          "headers": {
            "x-elastic-product-origin": "kibana",
            "user-agent": "Kibana/8.6.2",
            "authorization": "Basic ....",
            "x-opaque-id": "998ea841-e17e-46fe-b9bc-2b6839485374",
            "x-elastic-client-meta": "es=8.4.0p,js=16.18.1,t=8.2.0,hc=16.18.1"
          }
        },
        "id": 1
      },
      "name": "elasticsearch-js",
      "connection": {
        "url": "https://elasticsearch-es-http.services.svc:9200/",
        "id": "https://elasticsearch-es-http.services.svc:9200/",
        "headers": {},
        "status": "alive"
      },
      "attempts": 0,
      "aborted": false
    },
    "warnings": null
  }
}

Any advice how I can debug this or what can be wrong?