{"statusCode":503,"error":"Service Unavailable","message":"License is not available."}

Elastic Stack Kibana
1

Hello,

I have installed Elastic, but after 2-3 days stops working with the following error.
{"statusCode":503,"error":"Service Unavailable","message":"License is not available."}
Could you help me with investigation why its happaning ?

cat /etc/redhat-release
CentOS Linux release 7.9.2009 (Core)
bin/kibana --version
8.6.2
bin/elasticsearch --version
Version: 8.6.2, Build: rpm/2d58d0f136141f03239816a4e360a8d17b6d8f29/2023-02-13T09:35:20.314882762Z, JVM: 19.0.2

2

I also got this error when my actual elastic url was https://localhost:9200 and I set http://localhost:9200 in kibana.yml.
If you have SSH enabled in elasticsearch, check the url in kibana.yml and it should be https and https.
On the other side there can be other reasons to this. I have mentioned my case in which i got this error.

3

Thanks for your reply.

No, I didnt change it to https, I'm using http.
After rebooting of the machine works fine for several hours, but it fails in the end.
I'm able to see this error in kibana logs:

"License is not available, authentication is not possible.","log":{"level":"ERROR","logger":"plugins.security.authentication"},"process":{"pid":2414},"trace":{"id":"f447e50c1e57107daa15e35cd6939541"},"transaction":{"id":"8468a3a52ea55e95"}}

4

This means that there is some issue for Kibana to talk with Elasticsearch.

What is the result of a curl to your elasticsearch server from the kibana machine?

curl http://your-elastic-ip:9200

5

Hello,

with that command

curl  http://myiphere:9200
{"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":["Basic realm=\"security\" charset=\"UTF-8\"","ApiKey"]}}],"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":["Basic realm=\"security\" charset=\"UTF-8\"","ApiKey"]}},"status":401}

curl -u kibana_system:mypasswordhere http://myiphere:9200
{
  "name" : "elk01",
  "cluster_name" : "elk",
  "cluster_uuid" : "EW9pYnYfRLq5HKV5KXZ6JQ",
  "version" : {
    "number" : "8.6.2",
    "build_flavor" : "default",
    "build_type" : "rpm",
    "build_hash" : "2d58d0f136141f03239816a4e360a8d17b6d8f29",
    "build_date" : "2023-02-13T09:35:20.314882762Z",
    "build_snapshot" : false,
    "lucene_version" : "9.4.2",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}
6

Are Kibana and Elastic running on the same server?

If this request was from the Kibana server then it can talk to Elastic without any issues.

Did you change the default configuration? Per default version 8 will enable security and https, since you have http something was changed.

Can you share both your elasticsearch.yml and kibana.yml.

Also, share the logs of your Elasticsearch for the same time frame when you have the error in Kibana.

7

Yes, both are running and it works fine for a few hours after rebooting the machine.
Here is kibana.yml. I removed all unnecessary comments#

server.port: 5601
server.host: "myipishere"
server.name: "еlk01"
elasticsearch.hosts: ["http://myipishere:9200"]
elasticsearch.username: "kibana_system"
elasticsearch.password: "mypasswordhere"
logging:
  appenders:
    file:
      type: file
      fileName: /var/log/kibana/kibana.log
      layout:
        type: json
  root:
    appenders:
      - default
      - file
pid.file: /run/kibana/kibana.pid

here is the elastic.yml

cluster.name: bgelk
node.name: elk01
path.logs: /var/log/elasticsearch
network.host: ["localhost", "myipishere"]
http.port: 9200
discovery.type: single-node
xpack.security.enabled: true
xpack.security.enrollment.enabled: true
xpack.security.http.ssl:
  enabled: false
  keystore.path: certs/http.p12
xpack.security.transport.ssl:
  enabled: false
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12
http.host: 0.0.0.0

I have cleared the log because it was huge and hard to read. I will paste it once the issue is reproduced again.