Kibana URL is not redirecting to SAML login

I'm trying to configure SAML for my Elasticsearch cluster running in Kubernetes but when I access kibana URL it doesn't redirect me to the SAML login.

kibana configuration :-
kibana.yml: |
server:
name: "kibana"
host: 0.0.0.0
port: 5601
elasticsearch.url: "https://es-coordinating.gbi-gotu.svc.lb.usrno1.applecloud.io:9200"
xpack.security.enabled: true
server.ssl.enabled: true
server.ssl.key: /usr/share/kibana/config/tls_server/key.pem
server.ssl.certificate: /usr/share/kibana/config/tls_server/crt.pem
elasticsearch.ssl.certificateAuthorities: /usr/share/kibana/config/tls_server/crt.pem
xpack.security.authProviders: [saml]
server.xsrf.whitelist: [/api/security/v1/saml]

elasticsearch.yml :-
elasticsearch.yml: |
cluster.name: gbi-cluster
node.master: false
node.data: false
node.name: es-coordinating
node.ingest: false
network.host: 0.0.0.0
discovery.zen.minimum_master_nodes: 1
discovery.zen.ping.unicast.hosts: ["es-coordinating","es-master","es-data"]
node.ml: false
xpack.security.enabled: true
xpack.ml.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.key: /usr/share/elasticsearch/config/tls_server/key.pem
xpack.security.transport.ssl.certificate: /usr/share/elasticsearch/config/tls_server/crt.pem
xpack.security.transport.ssl.certificate_authorities: [ "/usr/share/elasticsearch/config/tls_server/crt.pem" ]
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.verification_mode: certificate
xpack.security.http.ssl.key: /usr/share/elasticsearch/config/tls_server/key.pem
xpack.security.http.ssl.certificate: /usr/share/elasticsearch/config/tls_server/crt.pem
xpack.security.http.ssl.certificate_authorities: [ "/usr/share/elasticsearch/config/tls_server/crt.pem" ]
xpack.security.authc.token.enabled: true
xpack.security.authc.realms.saml1:
type: saml
order: 2
idp.metadata.path: /usr/share/elasticsearch/config/saml/idp-metadata.xml
idp.entity_id: "https://idmsac-uat.corp.apple.com/IDMSWebAuth/SAMLLogin"
sp.entity_id: "https://gbikibana-gotu.corp.apple.com/"
sp.acs: "https://gbikibana-gotu.corp.apple.com/api/security/v1/saml"
sp.logout: "https://gbikibana-gotu.corp.apple.com/logout"
attributes.principal: "nameid:persistent"

Now when I try to access https://gbikibana-gotu.corp.apple.com -> it takes me directly to kibana app without any SAML redirection or authentication

SAML authentication is based on browser redirections and if you already have an active session with your IDP, the redirections might happen too quickly to notice and you end up in Kibana, authenticated via SAML , as expected.

Your current configuration and especially the

xpack.security.authProviders: [saml]

part, means that there is no other way for your user to be authenticated, so if you end up to Kibana as an authenticated user, you are probably authenticated with SAML.

If on the other hand

means that you end up to Kibana with an error, this , in turn, indicates that something in the configuration is wrong and the elasticsearch.log file will contain all the necessary information about the issue. This doc will probably be helpful, but we can also further assist if needed when we know what the problem is

Hi Ioannis,
Thanks for the response. I thought I will see kibana url redirecting to IDP. I couldn't find any relevant errors in elasticsearch logs. Please find below few exceptions which I'm seeing :-

[2019-03-11T07:07:48,243][WARN ][o.e.x.s.t.n.SecurityNetty4ServerTransport] [es-coordinating] exception caught on transport layer [NettyTcpChannel{localAddress=0.0.0.0/0.0.0.0:9300, remoteAddress=/17.99.232.26:58102}], closing connection
io.netty.handler.codec.DecoderException: io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record: 677177370d0a0d0a677177370d0a0d0a
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:459) ~[netty-codec-4.1.16.Final.jar:4.1.16.Final]
at java.lang.Thread.run(Thread.java:844) [?:?]
Caused by: io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record: 677177370d0a0d0a677177370d0a0d0a
[2019-03-11T07:08:00,251][WARN ][o.e.x.s.t.n.SecurityNetty4ServerTransport] [es-coordinating] exception caught on transport layer [NettyTcpChannel{localAddress=0.0.0.0/0.0.0.0:9300, remoteAddress=/17.99.232.26:40831}], closing connection
io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: no cipher suites in common

This has nothing to do with SAML . This is basically something that attempts to connect to the transport layer without using TLS and from the looks of it (677177370d0a0d0a677177370d0a0d0a decodes to "gqw7 gqw7") it's SecureWorlks doing a port scan.

Can you clarify: Are you redirected to Kibana and you are authenticated, or do you see an error in Kibana ?

I'm seeing kibana UI without any error and without any authentication also

Sorry, but this can't happen. If you are logged in to Kibana, you must be logged in as some user ( you have xpack.security.enabled: true ) so I can't understand the "without any authentication" part. What is shown next to the user icon on the lower left corner in Kibana, what is shown in the UI when you click on it ?

I don't see any usr icon on left side. I'm not using elasticsearch.user and elasticsearch.password field in kibana.yml . Also license is basic

Please see the screenshot

Security is not available in basic, see : https://www.elastic.co/subscriptions . You'd need a trial license or platinum to use SAML

okay let me try this

HI, I have updated the license to platinum also executed elasticsearch-setup-password and provided the elasticsearch.username and elasticsearch.password in kibana.yml but now I'm seeing error in kibana

{"type":"log","@timestamp":"2019-03-12T12:22:55Z","tags":["info","authentication"],"pid":1,"message":"Authentication attempt failed: [security_exception] Cannot find any matching realm for [SamlPrepareAuthenticationRequest{realmName=null, assertionConsumerServiceURL=https://0.0.0.0:5601/api/security/v1/saml}]"}
{"type":"error","@timestamp":"2019-03-12T12:22:55Z","tags":,"pid":1,"level":"error","error":{"message":"[security_exception] Cannot find any matching realm for [SamlPrepareAuthenticationRequest{realmName=null, assertionConsumerServiceURL=https://0.0.0.0:5601/api/security/v1/saml}]","name":"Error","stack":"[security_exception] Cannot find any matching realm for [SamlPrepareAuthenticationRequest{realmName=null, assertionConsumerServiceURL=https://0.0.0.0:5601/api/security/v1/saml}] :: {"path":"/_xpack/security/saml/prepare","query":{},"body":"{\"acs\":\"https://0.0.0.0:5601/api/security/v1/saml\"}","statusCode":500,"response":"{\"error\":{\"root_cause\":[{\"type\":\"security_exception\",\"reason\":\"Cannot find any matching realm for [SamlPrepareAuthenticationRequest{realmName=null, assertionConsumerServiceURL=https://0.0.0.0:5601/api/security/v1/saml}]\"}],\"type\":\"security_exception\",\"reason\":\"Cannot find any matching realm for [SamlPrepareAuthenticationRequest{realmName=null, assertionConsumerServiceURL=https://0.0.0.0:5601/api/security/v1/saml}]\"},\"status\":500}"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:307:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:266:7)\n at HttpConnector. (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:159:7)\n at IncomingMessage.bound (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/dist/lodash.js:729:21)\n at emitNone (events.js:111:20)\n at IncomingMessage.emit (events.js:208:7)\n at endReadableNT (_stream_readable.js:1064:12)\n at _combinedTickCallback (internal/process/next_tick.js:138:11)\n at process._tickDomainCallback (internal/process/next_tick.js:218:9)"},"url":{"protocol":null,"slashes":null,"auth":null,"host":null,"port":null,"hostname":null,"hash":null,"search":"","query":{},"pathname":"/","path":"/","href":"/"},"message":"[security_exception] Cannot find any matching realm for [SamlPrepareAuthenticationRequest{realmName=null, assertionConsumerServiceURL=https://0.0.0.0:5601/api/security/v1/saml}]"}

seems assertion url it is looking for is https://0.0.0.0:5601/api/security/v1/saml} but it should look for https://gbikibana-gotu.corp.apple.com/api/security/v1/saml

This is already covered in the link I shared above https://www.elastic.co/guide/en/elastic-stack-overview/current/trb-security-saml.html , please read through this ( section 1 )

I have added below lines to kibana.yml
server:
host: gbikibana-gotu.corp.apple.com
port: 5601
but my container is not coming up , getting the below error

{"type":"log","@timestamp":"2019-03-12T13:00:13Z","tags":["status","plugin:elasticsearch@6.3.0","info"],"pid":1,"state":"green","message":"Status changed from red to green - Ready","prevState":"red","prevMsg":"Request Timeout after 3000ms"}

{"type":"error","@timestamp":"2019-03-12T13:01:10Z","tags":["fatal"],"pid":1,"level":"fatal","error":{"message":"listen EADDRNOTAVAIL 17.99.220.96:5601","name":"Error","stack":"Error: listen EADDRNOTAVAIL 17.99.220.96:5601\n at Object._errnoException (util.js:1022:11)\n at _exceptionWithHostPort (util.js:1044:20)\n at Server.setupListenHandle [as _listen2] (net.js:1350:19)\n at listenInCluster (net.js:1408:12)\n at GetAddrInfoReqWrap.doListen (net.js:1517:7)\n at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:97:10)","code":"EADDRNOTAVAIL"},"message":"listen EADDRNOTAVAIL 17.99.220.96:5601"}
FATAL { Error: listen EADDRNOTAVAIL 17.99.220.96:5601
at Object._errnoException (util.js:1022:11)
at _exceptionWithHostPort (util.js:1044:20)
at Server.setupListenHandle [as _listen2] (net.js:1350:19)
at listenInCluster (net.js:1408:12)
at GetAddrInfoReqWrap.doListen (net.js:1517:7)
at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:97:10)
cause:
{ Error: listen EADDRNOTAVAIL 17.99.220.96:5601
at Object._errnoException (util.js:1022:11)
at _exceptionWithHostPort (util.js:1044:20)
at Server.setupListenHandle [as _listen2] (net.js:1350:19)
at listenInCluster (net.js:1408:12)
at GetAddrInfoReqWrap.doListen (net.js:1517:7)
at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:97:10)
code: 'EADDRNOTAVAIL',
errno: 'EADDRNOTAVAIL',
syscall: 'listen',
address: '17.99.220.96',
port: 5601 },
isOperational: true,
code: 'EADDRNOTAVAIL',
errno: 'EADDRNOTAVAIL',
syscall: 'listen',
address: '17.99.220.96',
port: 5601 }

Can you please share your entire kibana.yml ? Judging from the error message (EADDRNOTAVAIL 17.99.220.96:5601") it seems that kibana can't bind on that port on that IP

Please find below config file for kibana, I have tried with port 443, 5601 and 80 but getting same error

apiVersion: v1
kind: ConfigMap
metadata:
name: kibana
labels:
app: kibana
data:
kibana.yml: |
server:
host: gbikibana-gotu.corp.apple.com
port: 5601
elasticsearch.url: "https://es-coordinating.gbi-gotu.svc.lb.usrno1.applecloud.io:9200"
elasticsearch.username: kibana
elasticsearch.password: kibanapw
xpack.security.enabled: true
server.ssl.enabled: true
server.ssl.key: /usr/share/kibana/config/tls_server/key.pem
server.ssl.certificate: /usr/share/kibana/config/tls_server/crt.pem
elasticsearch.ssl.certificateAuthorities: /usr/share/kibana/config/tls_server/crt.pem
xpack.security.authProviders: [saml]
server.xsrf.whitelist: [/api/security/v1/saml]

As a general comment, please use the </> button or backticks (``` ) to format your messages and especially the configuration and logs parts, as this makes them so much easier to read.

I don't think it's the port that is at fault here, but rather the IP Address. You have configured it now with the addition of

server:
  host: gbikibana-gotu.corp.apple.com
  port: 5601

to try and bind to 17.99.220.96 (which I presume corresponds to gbikibana-gotu.corp.apple.com ) but it looks like this is not the IP address of the container. Can you verify that this address is indeed assigned to the container ?

I'm not perfectly sure about the Kubernetes networking part but maybe you should look into setting the

xpack.security.public:
  protocol: https
  hostname: gbikibana-gotu.corp.apple.com
  port: 443

stanza instead and leave the

server:
  host: 0.0.0.0

as it was.

If you go with the suggestion above, you need to also change your sp.acs both in the elasticsearch.yml configuration

sp.acs: "https://gbikibana-gotu.corp.apple.com:443/api/security/v1/saml"

and in the relevant configuration section of your SAML IdP. ( See the relevant issue on why this is currently needed )

Hey this worked and now I'm getting redirected to defined URL. Just one more thing is there any way by which I can configure kibana to give me a login prompt and redirects me after login is correct ?

Hi,

This is not at all how SAML Web Browser Single Sign On profile is supposed to work.. You can take a look at our blog post for a short introduction on how SAML works in general, but the general idea is that the authentication happens at the SAML Identity Provider and not at the SAML Service Provider ( The Elastic Stack in this case ) .

In fact, this specific property of SAML, the fact that the Service Provider never gets to see the user's credentials is one of the major benefits of using SAML ( or any other SSO solutions for that matter ).

If you want to authenticate your users in Kibana, maybe your use case is different and you should be looking at other authentication realms instead of SAML ?

I'm getting redirected to IDP (which shows that application u have selected does not exist) and doesn't take me to kibana app. I'm not seeing any error in kibana the problem seems to be from IDP team side that they haven't matched the idp.entity.ID to sp.entity_ID

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.