Trial license expires the next day after activation

Elastic Stack Elasticsearch
1

Hello everyone. Yesterday, I successfully deployed and configured Elasticsearch on Kubernetes. We need to use cross-cluster replication, so a trial period was activated. Everything was working fine for about 20 hours, then one node started randomly disconnecting from the master. It was suspected that it lacked resources, so I doubled them and updated the cluster. After that, the trial license became unavailable, and cross-cluster replication stopped working. This has happened twice now. We plan to purchase a platinum license, but I'm unsure about it because even the trial cannot last its full 30 days. It's all very strange; if we can't even try for a few days without having to recreate the clusters each time.
Elasticsearch version: 8.11.3
Logs when the node started randomly disconnecting:

{"@timestamp":"2024-03-19T08:55:31.341Z", "log.level": "INFO", "message":"node-join[{sfo3-es-sfo3-1}{RMyL7_UaTdaH54YrVrKclA}{fdZpfx5LRCegwNpxDstusg}{sfo3-es-sfo3-1}{10.244.6.107}{10.244.6.107:9300}{cdfhilmrstw}{8.11.3}{7000099-8500003} joining, removed [2.8s/2803ms] ago with reason [disconnected], [25] total removals], term: 1, version: 1076, delta: added {{sfo3-es-sfo3-1}{RMyL7_UaTdaH54YrVrKclA}{fdZpfx5LRCegwNpxDstusg}{sfo3-es-sfo3-1}{10.244.6.107}{10.244.6.107:9300}{cdfhilmrstw}{8.11.3}{7000099-8500003}}", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[sfo3-es-sfo3-0][masterService#updateTask][T#348]","log.logger":"org.elasticsearch.cluster.service.MasterService","elasticsearch.cluster.uuid":"w4SNQMnsSg2eWYXm51JQPQ","elasticsearch.node.id":"2QvL3D6eQnST1mQhbZEdpg","elasticsearch.node.name":"sfo3-es-sfo3-0","elasticsearch.cluster.name":"sfo3"}

Elasticsearch manifests:

---
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: sfo3
  namespace: elasticsearch
spec:
  version: 8.11.3
  http:
    tls:
      selfSignedCertificate:
        disabled: true
  transport:
    tls:
      certificateAuthorities:
        configMapName: remote-certs
    service:
      spec:
        type: LoadBalancer 
  nodeSets:
  - name: sfo3
    count: 2
    config:
      node.store.allow_mmap: False
    podTemplate:
      spec:
        containers:
        - name: elasticsearch
          resources:
            requests:
              memory: 2Gi
              cpu: 2
            limits:
              memory: 2Gi

Cluster real age:

kubectl get sts -n elasticsearch
NAME           READY   AGE
sfo3-es-sfo3   2/2     26h

API response to create follower index:

Error creating follower index: [403 Forbidden] {
  "error" : {
    "root_cause" : [
      {
        "type" : "security_exception",
        "reason" : "current license is non-compliant for [ccr]",
        "license.expired.feature" : "ccr"
      }
    ],
    "type" : "security_exception",
    "reason" : "current license is non-compliant for [ccr]",
    "license.expired.feature" : "ccr"
  },
  "status" : 403
}

Checking the license on the main cluster:

curl -u elastic:$PW -XGET http://$REGION-es-http:9200/_license
{
  "license" : {
    "status" : "active",
    "uid" : "918a3d9e-4e73-48b5-b970-0a38b2619db6",
    "type" : "basic",
    "issue_date" : "2024-03-19T10:15:12.734Z",
    "issue_date_in_millis" : 1710843312734,
    "max_nodes" : 1000,
    "max_resource_units" : null,
    "issued_to" : "sfo3",
    "issuer" : "elasticsearch",
    "start_date_in_millis" : -1
  }
}
curl -u elastic:$PW -X POST "http://$REGION-es-http:9200/_license/start_trial?acknowledge=true&pretty"
{
  "acknowledged" : true,
  "trial_was_started" : false,
  "error_message" : "Operation failed: Trial was already activated."
}

Checking the license on the replica cluster:

curl -u elastic:$PW -XGET http://$REGION-es-http:9200/_license
{
  "license" : {
    "status" : "active",
    "uid" : "423276a7-9e26-400e-8f57-cf1f3bc94a40",
    "type" : "trial",
    "issue_date" : "2024-03-18T07:51:38.770Z",
    "issue_date_in_millis" : 1710748298770,
    "expiry_date" : "2024-04-17T07:51:38.770Z",
    "expiry_date_in_millis" : 1713340298770,
    "max_nodes" : 1000,
    "max_resource_units" : null,
    "issued_to" : "fra1",
    "issuer" : "elasticsearch",
    "start_date_in_millis" : -1
  }
}

Let me know if I can provide any more important information, because it's just a development cluster without any clients. I hope for mutual assistance in resolving this issue. Thank you.

2

Hi @Sergey_Kuyanov

Is your "Main Cluster" brand new? To me it looks like the trial was activated/ expired at some point in the past. It can only be activated once.

Seems like When you added resources... There was perhaps an issue.. did you deactivate the trial then try to reactivate?

A new trial absolutely last 30 days.

3

Hello @stephenb ,
the primary and secondary clusters only had one day, the license was activated on both simultaneously, but on the primary cluster it worked for only one day, I completely rebuilt the cluster, and it seems now it's not dropping. But before that, it dropped two or three times.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.