Cannot register a fs snapshot repository in elastic 8.13.0

I installed elastic 8.13.0 in OCP 4.15.

I defined path.repo to ["/usr/share/elasticsearch/data/backup"]inelasticsearch.yml, but when I try to register a FS repository, I got error like"master_not_discovered_exception"`

sh-5.0$ export ELASTIC_PASSWORD="9TcoQN8x5cT641a1cvXR852U"
sh-5.0$ curl -X PUT "http://instana-es-http:9200/_snapshot/my_fs_backup?pretty" -H 'Content-Type: application/json' -u elastic:$ELASTIC_PASSWORD -d'
> {
>   "type": "fs",
>   "settings": {
>     "location": "/usr/share/elasticsearch/data/backup"
>   }
> }'
{
  "error" : {
    "root_cause" : [
      {
        "type" : "master_not_discovered_exception",
        "reason" : null
      }
    ],
    "type" : "master_not_discovered_exception",
    "reason" : null
  },
  "status" : 503
}

I created the elasticsearch CR in this way. I cannot un-comment node.master: true and node.data: true. Otherwise elastic pod will keep crash.

apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: instana
spec:
  version: 8.13.0
  nodeSets:
    - name: default
      count: 3
      config:
        # node.master: true
        # node.data: true
        # node.ingest: true
        node.store.allow_mmap: false
      podTemplate:
            spec:
              imagePullSecrets:
                - name: docker-registry
      volumeClaimTemplates:
        - metadata:
            name: elasticsearch-data # Do not change this name unless you set up a volume mount for the data path.
          spec:
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: 20Gi
  http:
    tls:
      selfSignedCertificate:
        disabled: true

Also I am not able to attach an NFS volume to elastic STS. The pod will not be created and describe sts said it was forbidden by security constraint in elastic installation.

Warning: would violate PodSecurity "restricted:v1.24": restricted volume types (volume "nfs-vol" uses restricted volume type "nfs"), seccompProfile (pod or containers "elastic-internal-init-filesystem", "elastic-internal-suspend", "elasticsearch" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")

  Warning  FailedCreate         109s (x3 over 109s)  statefulset-controller  create Pod instana-es-default-0 in StatefulSet instana-es-default failed error: pods "instana-es-default-0" is forbidden: unable to validate against any security context constraint: [provider "anyuid": Forbidden: not usable by user or serviceaccount, spec.volumes[1]: Invalid value: "nfs": nfs volumes are not allowed to be used, provider "restricted": Forbidden: not usable by user or serviceaccount, provider "nonroot-v2": Forbidden: not usable by user or serviceaccount, provider "nonroot": Forbidden: not usable by user or serviceaccount, provider "hostmount-anyuid": Forbidden: not usable by user or serviceaccount, provider "machine-api-termination-handler": Forbidden: not usable by user or serviceaccount, provider "hostnetwork-v2": Forbidden: not usable by user or serviceaccount, provider "hostnetwork": Forbidden: not usable by user or serviceaccount, provider "cassandra-scc": Forbidden: not usable by user or serviceaccount, provider "clickhouse-scc": Forbidden: not usable by user or serviceaccount, provider "hostaccess": Forbidden: not usable by user or serviceaccount, provider "rook-ceph": Forbidden: not usable by user or serviceaccount, provider "node-exporter": Forbidden: not usable by user or serviceaccount, provider "rook-ceph-csi": Forbidden: not usable by user or serviceaccount, provider "privileged": Forbidden: not usable by user or serviceaccount]

Your cluster has more fundamental problems than the ones related to snapshots. See these docs for more information:

If your cluster doesn’t have a stable master, many of its features won’t work correctly and Elasticsearch will report errors to clients and in its logs. You must fix the master node’s instability before addressing these other issues. It will not be possible to solve any other issues while there is no elected master node or the elected master node is unstable.

1 Like

I un-comment the node.master: true and node.data: true in my CR, then I got these logs in my pod logs.

it says that unknown setting [node.data] please check that any required plugins are installed. Is it true there is NO plugins in elastic v8.13.x?

And is there any document tell me what exactly items should be in a elastic CR and what exactly items should be in elasticsearch.yaml ?

{"@timestamp":"2024-04-19T09:51:11.378Z", "log.level":"ERROR", "message":"fatal exception while booting Elasticsearch", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.bootstrap.Elasticsearch","elasticsearch.node.name":"instana-es-default-0","elasticsearch.cluster.name":"instana","error.type":"java.lang.IllegalArgumentException","error.message":"unknown setting [node.data] please check that any required plugins are installed, or check the breaking changes documentation for removed settings","error.stack_trace":"java.lang.IllegalArgumentException: unknown setting [node.data] please check that any required plugins are installed, or check the breaking changes documentation for removed settings\n\tat org.elasticsearch.server@8.13.0/org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:563)\n\tat org.elasticsearch.server@8.13.0/org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:509)\n\tat org.elasticsearch.server@8.13.0/org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:479)\n\tat org.elasticsearch.server@8.13.0/org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:449)\n\tat org.elasticsearch.server@8.13.0/org.elasticsearch.common.settings.SettingsModule.<init>(SettingsModule.java:132)\n\tat org.elasticsearch.server@8.13.0/org.elasticsearch.common.settings.SettingsModule.<init>(SettingsModule.java:50)\n\tat org.elasticsearch.server@8.13.0/org.elasticsearch.node.NodeConstruction.validateSettings(NodeConstruction.java:494)\n\tat org.elasticsearch.server@8.13.0/org.elasticsearch.node.NodeConstruction.prepareConstruction(NodeConstruction.java:255)\n\tat org.elasticsearch.server@8.13.0/org.elasticsearch.node.Node.<init>(Node.java:192)\n\tat org.elasticsearch.server@8.13.0/org.elasticsearch.bootstrap.Elasticsearch$2.<init>(Elasticsearch.java:237)\n\tat org.elasticsearch.server@8.13.0/org.elasticsearch.bootstrap.Elasticsearch.initPhase3(Elasticsearch.java:237)\n\tat org.elasticsearch.server@8.13.0/org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:74)\n\tSuppressed: java.lang.IllegalArgumentException: unknown setting [node.master] please check that any required plugins are installed, or check the breaking changes documentation for removed settings\n\t\t... 12 more\n"}
ERROR: Elasticsearch did not exit normally - check the logs at /usr/share/elasticsearch/logs/instana.log

ERROR: Elasticsearch died while starting up, with exit code 1

If I un-comment node.master: true only, the pod log will said unknown setting [node.master]

{"@timestamp":"2024-04-19T09:57:21.805Z", "log.level":"ERROR", "message":"fatal exception while booting Elasticsearch", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.bootstrap.Elasticsearch","elasticsearch.node.name":"instana-es-default-0","elasticsearch.cluster.name":"instana","error.type":"java.lang.IllegalArgumentException","error.message":"unknown setting [node.master] please check that any required plugins are installed, or check the breaking changes documentation for removed settings","error.stack_trace":"java.lang.IllegalArgumentException: unknown setting [node.master] please check that any required plugins are installed, or check the breaking changes documentation for removed settings\n\tat org.elasticsearch.server@8.13.0/org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:563)\n\tat org.elasticsearch.server@8.13.0/org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:509)\n\tat org.elasticsearch.server@8.13.0/org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:479)\n\tat org.elasticsearch.server@8.13.0/org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:449)\n\tat org.elasticsearch.server@8.13.0/org.elasticsearch.common.settings.SettingsModule.<init>(SettingsModule.java:132)\n\tat org.elasticsearch.server@8.13.0/org.elasticsearch.common.settings.SettingsModule.<init>(SettingsModule.java:50)\n\tat org.elasticsearch.server@8.13.0/org.elasticsearch.node.NodeConstruction.validateSettings(NodeConstruction.java:494)\n\tat org.elasticsearch.server@8.13.0/org.elasticsearch.node.NodeConstruction.prepareConstruction(NodeConstruction.java:255)\n\tat org.elasticsearch.server@8.13.0/org.elasticsearch.node.Node.<init>(Node.java:192)\n\tat org.elasticsearch.server@8.13.0/org.elasticsearch.bootstrap.Elasticsearch$2.<init>(Elasticsearch.java:237)\n\tat org.elasticsearch.server@8.13.0/org.elasticsearch.bootstrap.Elasticsearch.initPhase3(Elasticsearch.java:237)\n\tat org.elasticsearch.server@8.13.0/org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:74)\n"}

I got clue. When I installed elasticsearch, it works fine and elected master.

then when I changed elasticsearch.yml to. add path.repo by change the secret by which elasticsearch.yml mounted, then restarted all the 3 elastic pod, all pods keep claim no master been discovered.


{"@timestamp":"2024-04-19T10:23:28.349Z", "log.level": "WARN", "message":"master not discovered or elected yet, an election requires at least 2 nodes with ids from [NXbb5SLjRW6zTsk5w3_Hqg, cgMQ3BlcSASi0Abi-do6cg, 9F7Mj-brSxmqMINmgtf8Sg], have only discovered non-quorum [{instana-es-default-0}{9F7Mj-brSxmqMINmgtf8Sg}{shoCtvUDTKef6e1Uzategw}{instana-es-default-0}{10.254.32.59}{10.254.32.59:9300}{cdfhilmrstw}{8.13.0}{7000099-8503000}]; discovery will continue using [10.254.12.53:9300, 10.254.28.54:9300, 10.254.32.58:9300] from hosts providers and [{instana-es-default-0}{9F7Mj-brSxmqMINmgtf8Sg}{shoCtvUDTKef6e1Uzategw}{instana-es-default-0}{10.254.32.59}{10.254.32.59:9300}{cdfhilmrstw}{8.13.0}{7000099-8503000}] from last-known cluster state; node term 1, last-accepted version 55 in term 1; for troubleshooting guidance, see https://www.elastic.co/guide/en/elasticsearch/reference/8.13/discovery-troubleshooting.html", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[instana-es-default-0][cluster_coordination][T#1]","log.logger":"org.elasticsearch.cluster.coordination.ClusterFormationFailureHelper","elasticsearch.node.name":"instana-es-default-0","elasticsearch.cluster.name":"instana"}

waiting for a while, I can find these messages from pod log

{"@timestamp":"2024-04-19T10:34:18.937Z", "log.level": "WARN", "message":"address [10.254.12.53:9300], node [unknown discovery result: [][10.254.12.53:9300] connect_timeout[30s]; for summary, see logs from org.elasticsearch.cluster.coordination.ClusterFormationFailureHelper; for troubleshooting guidance, see https://www.elastic.co/guide/en/elasticsearch/reference/8.13/discovery-troubleshooting.html", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[instana-es-default-0][generic][T#6]","log.logger":"org.elasticsearch.discovery.PeerFinder","elasticsearch.node.name":"instana-es-default-0","elasticsearch.cluster.name":"instana"}

{"@timestamp":"2024-04-19T10:34:18.937Z", "log.level": "WARN", "message":"address [10.254.28.54:9300], node [unknown discovery result: [][10.254.28.54:9300] connect_timeout[30s]; for summary, see logs from org.elasticsearch.cluster.coordination.ClusterFormationFailureHelper; for troubleshooting guidance, see https://www.elastic.co/guide/en/elasticsearch/reference/8.13/discovery-troubleshooting.html", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[instana-es-default-0][generic][T#5]","log.logger":"org.elasticsearch.discovery.PeerFinde
r","elasticsearch.node.name":"instana-es-default-0","elasticsearch.cluster.name":"instana"}

{"@timestamp":"2024-04-19T10:34:18.937Z", "log.level": "WARN", "message":"address [10.254.32.58:9300], node [unknown discovery result: [][10.254.32.58:9300] connect_timeout[30s]; for summary, see logs from org.elasticsearch.cluster.coordination.ClusterFormationFailureHelper; for troubleshooting guidance, see https://www.elastic.co/guide/en/elasticsearch/reference/8.13/discovery-troubleshooting.html", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[instana-es-default-0][generic][T#3]","log.logger":"org.elasticsearch.discovery.PeerFinde
r","elasticsearch.node.name":"instana-es-default-0","elasticsearch.cluster.name":"instana"}

but there was no those IPs actually after restart

oc get po -o wide
NAME                   READY   STATUS      RESTARTS   AGE    IP             NODE                NOMINATED NODE   READINESS GATES
curl                   0/1     Completed   0          2d1h   10.254.16.27   worker2.o1-729588   <none>           <none>
instana-es-default-0   1/1     Running     0          10m    10.254.32.60   worker3.o1-729588   <none>           <none>
instana-es-default-1   1/1     Running     0          10m    10.254.28.56   worker4.o1-729588   <none>           <none>
instana-es-default-2   1/1     Running     0          10m    10.254.12.55   worker0.o1-729588   <none>           <none>

I tried another approach. I scale the elastic sts to 0 replicas, then scale it to 3, but elastic pod still looking to the old IP addresses for nodes. That should be a problem.

[root@api.o1-729588 ~]# oc scale sts instana-es-default --replicas=0
statefulset.apps/instana-es-default scaled
[root@api.o1-729588 ~]# oc scale sts instana-es-default --replicas=3
statefulset.apps/instana-es-default scaled
[root@api.o1-729588 ~]#
[root@api.o1-729588 ~]# oc get po -o wide
NAME                   READY   STATUS      RESTARTS   AGE    IP             NODE                                NOMINATED NODE   READINESS GATES
curl                   0/1     Completed   0          2d1h   10.254.16.27   worker2.o1-729588   <none>           <none>
instana-es-default-0   1/1     Running     0          108s   10.254.32.62   worker3.o1-729588   <none>           <none>
instana-es-default-1   1/1     Running     0          108s   10.254.12.58   worker0.o1-729588   <none>           <none>
instana-es-default-2   1/1     Running     0          108s   10.254.28.58   worker4.o1-729588   <none>           <none>
[root@api.o1-729588 ~]#
{"@timestamp":"2024-04-19T11:04:41.788Z", "log.level": "WARN", "message":"address [10.254.12.57:9300], node [unknown discovery result: [][10.254.12.57:9300] connect_timeout[30s]; for summary, see logs from org.elasticsearch.cluster.coordination.ClusterFormationFailureHelper; for troubleshooting guidance, see https://www.elastic.co/guide/en/elasticsearch/reference/8.13/discovery-troubleshooting.html", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[instana-es-default-0][generic][T#3]","log.logger":"org.elasticsearch.discovery.PeerFinder","elasticsearch.node.name":"instana-es-default-0","elasticsearch.cluster.name":"instana"}
{"@timestamp":"2024-04-19T11:04:41.788Z", "log.level": "WARN", "message":"address [10.254.32.61:9300], node [unknown discovery result: [][10.254.32.61:9300] connect_timeout[30s]; for summary, see logs from org.elasticsearch.cluster.coordination.ClusterFormationFailureHelper; for troubleshooting guidance, see https://www.elastic.co/guide/en/elasticsearch/reference/8.13/discovery-troubleshooting.html", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[instana-es-default-0][generic][T#2]","log.logger":"org.elasticsearch.discovery.PeerFinder","elasticsearch.node.name":"instana-es-default-0","elasticsearch.cluster.name":"instana"}
{"@timestamp":"2024-04-19T11:04:41.788Z", "log.level": "WARN", "message":"address [10.254.28.57:9300], node [unknown discovery result: [][10.254.28.57:9300] connect_timeout[30s]; for summary, see logs from org.elasticsearch.cluster.coordination.ClusterFormationFailureHelper; for troubleshooting guidance, see https://www.elastic.co/guide/en/elasticsearch/reference/8.13/discovery-troubleshooting.html", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[instana-es-default-0][generic][T#5]","log.logger":"org.elasticsearch.discovery.PeerFinder","elasticsearch.node.name":"instana-es-default-0","elasticsearch.cluster.name":"instana"}
{"@timestamp":"2024-04-19T11:04:50.706Z", "log.level": "WARN", "message":"master not discovered or elected yet, an election requires at least 2 nodes with ids from [DXo4BIiJTRSUOiGCCDy9Zw, 8zZQRMjhQRq8zdzW7CEgoQ, l8imZI5dSOKbHNCIIo_i_A], have only discovered non-quorum [{instana-es-default-0}{l8imZI5dSOKbHNCIIo_i_A}{2pluVwLARqGKTcfo7P1Gvg}{instana-es-default-0}{10.254.32.62}{10.254.32.62:9300}{cdfhilmrstw}{8.13.0}{7000099-8503000}]; discovery will continue using [10.254.12.57:9300, 10.254.28.57:9300, 10.254.32.61:9300] from hosts providers and [{instana-es-default-0}{l8imZI5dSOKbHNCIIo_i_A}{2pluVwLARqGKTcfo7P1Gvg}{instana-es-default-0}{10.254.32.62}{10.254.32.62:9300}{cdfhilmrstw}{8.13.0}{7000099-8503000}] from last-known cluster state; node term 3, last-accepted version 58 in term 3; for troubleshooting guidance, see https://www.elastic.co/guide/en/elasticsearch/reference/8.13/discovery-troubleshooting.html", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[instana-es-default-0][cluster_coordination][T#1]","log.logger":"org.elasticsearch.cluster.coordination.ClusterFormationFailureHelper","elasticsearch.node.name":"instana-es-default-0","elasticsearch.cluster.name":"instana"}

That's right, node.data and node.master aren't settings any more. See these docs for more information.