Can't log in in Kibana GUI

Dear all,

I need your support.

I have installed ES 7.5, Kibana 7.5

I have switched license type to trial in order to use x-pack.

Also, i have changed all built-in users passwords.

Currently no one user can log in Kibana GUI.

Here are ES and Kibana configs:

ES:

path.data: /d02/elastic/data

Path to log files:

path.logs: /d02/elastic/logs

bootstrap.system_call_filter: false

#bootstrap.seccomp: false

bootstrap.memory_lock: true

network.host: "xx.x.xxx.xx"

Set a custom port for HTTP:

http.port: 9200

discovery.type: single-node

index.codec: best_compression

#index.number_of_shards: 1

xpack.security.enabled: true

Kibana

server.port: 5601

server.host: "xx.x.xx.xx"

server.defaultRoute: "/etc/kibana"

server.maxPayloadBytes: 104857600

server.name: "kibana"

elasticsearch.hosts: "http://xx.x.xx.xx:9200"

elasticsearch.preserveHost: true

kibana.index: ".kibana"

kibana.defaultAppId: "discover"

elasticsearch.username: "kibana"
elasticsearch.password: "YYYYYYYY"

elasticsearch.pingTimeout: 1500

elasticsearch.requestTimeout: 30000

elasticsearch.requestHeadersWhitelist:

elasticsearch.customHeaders: {}

elasticsearch.shardTimeout: 0

elasticsearch.startupTimeout: 5000

pid.file: /var/run/kibana.pid

logging.dest: /var/log/kibana/kibana.log

logging.silent: false

logging.quiet: false

logging.verbose: true

ops.interval: 5000

---------------------------------- X-Pack ------------------------------------

xpack.security.enabled: true

xpack.security.sessionTimeout: 600000

Also some superusers created by me and they can not login Kibana too.

What is the reason i can not log in Kibana for built in users and other ones?

@emilio have you instructed all users to clear their browser cookies? I'd also try restarting Kibana.

Yes, they have cleared cookies and also tried to login using different browser. The same problem.

When you curl elasticsearch with basic auth (your elastic username and password) do you get the elasticsearch default http response from Elasticsearch?

curl -u elastic:changeme localhost:9200

If not, reset your built in users and try again. If you can Auth successfully. Make sure you set the working creds in kibana.yml

curl -u elastic:changeme xx.xx.xxx.xx:9200

 {"error":{"root_cause":[{"type":"security_exception","reason":"failed to authenticate user [elastic]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}}],"type":"security_exception","reason":"failed to authenticate user [elastic]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}},"status":401}

here is request using built in users with changed passwords

```
curl -u elastic:elasticnewpassword xx.xx.xxx.xx:9200

{
  "name" : "xxxxxxx.yyyyyyyyyy.com",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "Q2JPHfYnSPKWWqgn7P6FVA",
  "version" : {
    "number" : "7.5.0",
    "build_flavor" : "default",
    "build_type" : "rpm",
    "build_hash" : "e9ccaed468e2fac2275a3761849cbee64b39519f",
    "build_date" : "2019-11-26T01:06:52.518245Z",
    "build_snapshot" : false,
    "lucene_version" : "8.3.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}
```

Also i receive such kind of kibana logs in /var/log/kibana:

 "type":"log","@timestamp":"2019-12-28T11:04:25Z","tags":["debug","http","server","Kibana","cookie-session-storage"],"pid":28206,"message":"Error: Unauthorized"}
{"type":"log","@timestamp":"2019-12-28T11:04:25Z","tags":["debug","plugins","security","basic"],"pid":28206,"message":"Trying to perform a login."}
{"type":"log","@timestamp":"2019-12-28T11:04:25Z","tags":["debug","plugins","security","basic"],"pid":28206,"message":"Failed to perform a login: [security_exception] missing authentication credentials for REST request [/_security/_authenticate], with { header={ WWW-Authenticate=\"Basic realm=\\\"security\\\" charset=\\\"UTF-8\\\"\" } }"}
{"type":"log","@timestamp":"2019-12-28T11:04:25Z","tags":["license","debug","xpack"],"pid":28206,"message":"Calling [data] Elasticsearch _xpack API. Polling frequency: 30001"}
{"type":"response","@timestamp":"2019-12-28T11:04:25Z","tags":[],"pid":28206,"method":"post","statusCode":401,"req":{"url":"/api/security/v1/login","method":"post","headers":{"host":"10.0.139.79:5601","connection":"keep-alive","content-length":"43","accept":"application/json, text/plain, */*","origin":"http://10.0.139.79:5601","kbn-version":"7.5.0","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36","content-type":"application/json;charset=UTF-8","referer":"http://10.0.139.79:5601/login?next=%2F","accept-encoding":"gzip, deflate","accept-language":"en-US,en;q=0.9"},"remoteAddress":"10.0.206.38","userAgent":"10.0.206.38","referer":"http://10.0.139.79:5601/login?next=%2F"},"res":{"statusCode":401,"responseTime":27,"contentLength":9},"message":"POST /api/security/v1/login 401 27ms - 9.0B"}
{"type":"log","@timestamp":"2019-12-28T11:04:25Z","tags":["debug","monitoring","kibana-monitoring"],"pid":28206,"message":"Received Kibana Ops event data"}
{"type":"log","@timestamp":"2019-12-28T11:04:25Z","tags":["debug","monitoring","kibana-monitoring"],"pid":28206,"message":"Received Kibana Ops event data"}
{"type":"log","@timestamp":"2019-12-28T11:04:25Z","tags":["debug","stats-collection"],"pid":28206,"message":"Fetching data from kibana_stats collector"}
{"type":"log","@timestamp":"2019-12-28T11:04:25Z","tags":["debug","stats-collection"],"pid":28206,"message":"Fetching data from kibana_settings collector"}
{"type":"log","@timestamp":"2019-12-28T11:04:25Z","tags":["debug","stats-collection"],"pid":28206,"message":"not sending [kibana_settings] monitoring document because [undefined] is null or invalid."}
{"type":"log","@timestamp":"2019-12-28T11:04:25Z","tags":["debug","monitoring","kibana-monitoring"],"pid":28206,"message":"Uploading bulk stats payload to the local cluster"}
{"type":"log","@timestamp":"2019-12-28T11:04:25Z","tags":["debug","monitoring","kibana-monitoring"],"pid":28206,"message":"Uploaded bulk stats payload to the local cluster"}
{"type":"log","@timestamp":"2019-12-28T11:04:26Z","tags":["plugin","debug"],"pid":28206,"message":"Checking Elasticsearch version"}
{"type":"ops","@timestamp":"2019-12-28T11:04:26Z","tags":[],"pid":28206,"os":{"load":[0.189453125,0.712890625,0.75439453125],"mem":{"total":25196810240,"free":1758330880},"uptime":9432669},"proc":{"uptime":111.582,"mem":{"rss":347217920,"heapTotal":260194304,"heapUsed":212437248,"external":1790392},"delay":0.12874412536621094},"load":{"requests":{"5601":{"total":1,"disconnects":0,"statusCodes":{"401":1}}},"responseTimes":{"5601":{"avg":27,"max":27}},"sockets":{"http":{"total":3,"169.254.169.254:80:":3},"https":{"total":0}}},"message":"memory: 202.6MB uptime: 0:01:52 load: [0.19 0.71 0.75] delay: 0.129"}
^C{"type":"log","@timestamp":"2019-12-28T11:04:28Z","tags":["plugin","debug"],"pid":28206,"message":"Checking Elasticsearch version"}

Thanks. I see you can auth successfully to Elasticsearch so your keystore is working.

Kibana logs: missing authentication credentials for REST request (See last sentence).

Place the elastic:elasticnewpassword credentials in kibana.yml under elasticsearch.username & elasticsearch.password.

Your Kibana requests are not being passed with authentication, hence the 401. Hope that helps.

As you can see from kibana.yml, i have defined ES username and password not using keystore, but adding into kibana.yml

But not working at all

Okay, sounds like you have another issue somewhere.

Can you follow the below article to the T and see if it works?

If you're unable to get it to work with the help of this article, you have another issue in your environment that is outside the scope of this issue post.

@emilio

Any luck so far?