Upgrading Elasticsearch and Kibana running on Kubernetes from 7.12.0 to 7.17.10

I am trying to upgrade my cluster to latest supported image and I am having issues.
According to Elastic documentation I should be able to upgrade directly from any version between 7.0–7.16 to 7.17.10, yet when I tried that in my non-prod environment, the nodes showed the following in the logs:

{"type": "server", "timestamp": "2023-06-14T11:51:56,132Z", "level": "INFO", "component": "o.e.x.s.a.RealmsAuthenticator", "cluster.name": "elasticsearch", "node.name": "elasticsearch-master-0", "message": "Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]", "cluster.uuid": "", "node.id": "" }

{"type": "server", "timestamp": "2023-06-14T11:52:06,130Z", "level": "ERROR", "component": "o.e.x.s.a.e.ReservedRealm", "cluster.name": "elasticsearch", "node.name": "elasticsearch-master-0", "message": "failed to retrieve password hash for reserved user [elastic]", "cluster.uuid": "", "node.id": ""

I didn't apply any changes in configuration, simply swapped the image version reference from 7.12.0 to 7.17.10 and that's what I got a few minutes later.

my elasticsearch.yml looks like this:

path.repo: /usr/share/elasticsearch/backup
path.logs: /usr/share/elasticsearch/logs/elk
reindex.remote.whitelist: `["****, *****" ]`
http.max_content_length: 1000mb
xpack:
  security:
    enabled: true
    transport:
      ssl:
        enabled: true
        verification_mode: certificate
        key: /usr/share/elasticsearch/config/certs/tls.key
        certificate: /usr/share/elasticsearch/config/certs/tls.crt
        certificate_authorities:  /usr/share/elasticsearch/config/certs/tls.crt
    http:
      ssl:
        enabled: true
        verification_mode: certificate
        key: /usr/share/elasticsearch/config/certs/tls.key
        certificate: /usr/share/elasticsearch/config/certs/tls.crt
        certificate_authorities:  /usr/share/elasticsearch/config/certs/tls.crt
    authc: 
      realms:
        native:
          native1:
            order: 0
        oidc:
          oidc1:
            rp.client_id: ""
            rp.response_type: "code" 
            rp.requested_scopes: ["openid", "email", "profile"]
            rp.redirect_uri: "https://********/api/security/v1/oidc"
            op.issuer: "https://login.microsoftonline.com/****/v2.0" 
            op.authorization_endpoint: "https://login.microsoftonline.com/******/oauth2/v2.0/authorize" 
            op.token_endpoint: "https://login.microsoftonline.com/******/oauth2/v2.0/token" 
            op.userinfo_endpoint: "https://graph.microsoft.com/oidc/userinfo" 
            op.endsession_endpoint: "https://login.microsoftonline.com/******/oauth2/v2.0/logout" 
            rp.post_logout_redirect_uri: "https://******/logged_out" 
            op.jwkset_path: "https://login.microsoftonline.com/******/discovery/v2.0/keys" 
            claims.principal: email 
            claims.groups: groups
            claim_patterns.principal: "^([^@]+)@.*$"
xpack.monitoring.collection.interval: 600s
xpack.monitoring.collection.index.stats.timeout: 120s
xpack.monitoring.collection.node.stats.timeout: 120s
xpack.monitoring.collection.cluster.stats.timeout: 120s
indices.query.bool.max_clause_count: 3000

Please let me know if more information is needed and if there's something wrong or missing with what I tried to do.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.