Elasticsearch SAML integration while using HTTP-Redirect

Elastic Stack Elasticsearch
1

I am integrating ELK with a AD serve via SAML, the partial message from metadata file supplied by AD serve is
{

</ds:X509Data></ds:KeyInfo></md:KeyDescriptor>md:NameIDFormaturn:oasis:names:tc:SAML:1.1:nameid-format:unspecified</md:NameIDFormat><md:SingleSignOnService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST"
}

but when i login in face a problem that "Cannot find [{urn:oasis:names:tc:SAML:2.0:metadata}IDPSSODescriptor]/[urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect] in descriptor"

and i search in google , i saw
• An with an entityID that matches the {es} configuration
• An that supports the SAML 2.0 protocol (urn:oasis:names:tc:SAML:2.0:protocol).
• At least one that is configured for signing (that is, it has use="signing" or leaves the use unspecified)
• A with binding of HTTP-Redirect (urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect)
• If you wish to support Single Logout, a with binding of HTTP-Redirect (urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect)
Here is the link to describe it.
https://github.com/elastic/elasticsearch/blob/master/x-pack/docs/en/security/authentication/saml-guide.asciidoc.

As we need SSO, So HTTP-Redirect is necessary.
my question is
1 if we use SSO, Do AD server have to support HTTP-Redirect?
2 if the answer is yes ,why.
3 can i disable SSO temporarily. how?

2

When you say "AD", I assume you mean ADFS - is that correct?

ADFS can support HTTP-Redirect and has been successfully used with the Elastic Stack. You may need to look at configuration options for your AD install.

1 if we use SSO, Do AD server have to support HTTP-Redirect?

Your Identity Provider must support a HTTP-Redirect binding for sign on, yes.
It's OK if it supports additional bindings, but we require a Redirect binding.

2 if the answer is yes ,why.

Because implementing support for each binding type is additional work, and we have chosen to implement HTTP-Redirect support, because it is required by the SAML comformance spec and all interoperability profiles.

3 can i disable SSO temporarily. how?

You can turn off SAML support in Kibana.

See: "SingleSignOnService Binding HTTP-POST" for previous discussion on this topic.

3

HI Tim
Thank you very much.

4

HI Tim
I have another question。
do you have tetative time to get POST Binding implemented? or you wont implement anytime?

5

There is no current plan to implement POST bindings.
We may do it if there is sufficient demand, but we haven't seen enough requests for it to make it a priority right now.

6

HI Tim
I face a issue that when i enter the home page url of ELK in my environment, the browser return
“
{"statusCode":401,"error":"Unauthorized","message":"[security_exception] unable to authenticate user [] for action [cluster:admin/xpack/security/saml/authenticate], with { header={ WWW-Authenticate="Basic realm=\"security\" charset=\"UTF-8\"" } } :: {"path":"/_xpack/security/saml/authenticate","query":{},"body":"{\"ids\":[],\"content\":\"PHNhbWxwOlJlc3BvbnNlIFZlcnNpb249IjIuMCIgSUQ9Ik9FVHM3WmJ2am53MlcyR3B3VWhqWFNtMmI4eSIgSXNzdWVJbnN0YW50PSIyMDE4LTA3LTE3VDAwOjQxOjM1Ljg0NFoiIEluUmVzcG9uc2VUbz0iX2UxOTk0OTI4NTY1OTI5NGZkOWQxMTAzNDAyYTZhOTdhMjI4YjUzY2IiIERlc3RpbmF0aW9uPSJodHRwczovL3NlbnQyLWtpYmFuYS1kZXYuZHNlbnRpZW5jZS5uZXQ6NDQzL2FwaS9zZWN1cml0eS92MS9zYW1sIiB4bWxuczpzYW1scD0idXJuOm9hc2lzOm5hbWVzOnRjOlNBTUw6Mi4wOnByb3RvY29sIj48c2FtbDpJc3N1ZXIgeG1sbnM6c2FtbD0idXJuOm9hc2lzOm5hbWVzOnRjOlNBTUw6Mi4wOmFzc2VydGlvbiI+aHR0cHM6Ly9xY3dhLmhvbmV5d2VsbC5jb20vc2FtbDI8L3NhbWw6SXNzdWVyPjxkczpTaWduYXR1cmUgeG1sbnM6ZHM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvMDkveG1sZHNpZyMiPgo8ZHM6U2lnbmVkSW5mbz4KPGRzOkNhbm9uaWNhbGl6YXRpb25NZXRob2QgQWxnb3JpdGhtPSJodHRwOi8vd3d3LnczLm9yZy8yMDAxLzEwL3htbC1leGMtYzE0biMiLz4KPGRzOlNpZ25hdHVyZU1ldGhvZCBBbGdvcml0aG09Imh0dHA6Ly93d3cudzMub3JnLzIwMDEvMDQveG1sZHNpZy1tb3JlI3JzYS1zaGEyNTYiLz4KPGRzOlJlZmVyZW5jZSBVUkk9IiNPRVRzN1pidmpudzJXMkdwd1VoalhTbTJiOHkiPgo8ZHM6VHJhbnNmb3Jtcz4KPGRzOlRyYW5zZm9ybSBBbGdvcml0aG09Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvMDkveG1sZHNpZyNlbnZlbG9wZWQtc2lnbmF0dXJlIi8+CjxkczpUcmFuc2Zvcm0gQWxnb3JpdGhtPSJodHRwOi8vd3d3LnczLm9yZy8yMDAxLzEwL3htbC1leGMtYzE0biMiLz4KPC9kczpUcmFuc2Zvcm1zPgo8ZHM6RGlnZXN0TWV0aG9kIEFsZ29yaXRobT0iaHR0cDovL3d3dy53My5vcmcvMjAwMS8wNC94bWxlbmMjc2hhMjU2Ii8+CjxkczpEaWdlc3RWYWx1ZT5HTFlGejdQdTM4elFXTzl2SzNxU293Y0lKUXltOGJNUWNIWlMzS3F0cWlRPTwvZHM6RGlnZXN0VmFsdWU+CjwvZHM6UmVmZXJlbmNlPgo8L2RzOlNpZ25lZEluZm8+CjxkczpTaWduYXR1cmVWYWx1ZT4Kd3M3aFVZSUpEU2tOV3JDVkNQTkc2TTFhZHFwVzdiRjl5aXN2YmdleC96Qy9uTU9LaFdTV1BDQ0pXTlhsYW82aFhycWFZRnB1OVdodgptTXJQdHBnODJUQUZ6QWpqQXllczFidWtpS1kvM3d3aVZWdklnMUc4UkRONzR3dWlzTWt6aTE2MmtibUtXZWV5aVMwL3RTeFEzRkpQCk8xNkFUOXhDSEZWSTlyNlFNWmoxbjdRTUZjM29IKzVkTUVSODFYclpJNXhkS042UmpBTFV6Vlh0aDNwdUJaTTFZK1dLek9Db3Y5YTIKZjg1UlZ1a0VWczR2YWRhUTh0SURmZ1hVeUhjYVdiWTRzL09IaVdMeTMrMXRzSXczRlNobG9yQXlkQWZmRFdwUjBsRFZZTEdXK3BscApvS3hmYlQ5S0tqNFplTm1FYkZ0bFFPa3VlUFR4ZkJaVjhxSVFSdz09CjwvZHM6U2lnbmF0dXJlVmFsdWU+CjwvZHM6U2lnbmF0dXJlPjxzYW1scDpTdGF0dXM+PHNhbWxwOlN0YXR1c0NvZGUgVmFsdWU9InVybjpvYXNpczpuYW1lczp0YzpTQU1MOjIuMDpzdGF0dXM6UmVxdWVzdGVyIi8+PHNhbWxwOlN0YXR1c01lc3NhZ2U+U2lnbmF0dXJlIHJlcXVpcmVkPC9zYW1scDpTdGF0dXNNZXNzYWdlPjwvc2FtbHA6U3RhdHVzPjwvc2FtbHA6UmVzcG9uc2U+\"}","statusCode":401,"response":"{\"error\":{\"root_cause\":[{\"type\":\"security_exception\",\"reason\":\"unable to authenticate user [] for action [cluster:admin/xpack/security/saml/authenticate]\",\"header\":{\"WWW-Authenticate\":\"Basic realm=\\\"security\\\" charset=\\\"UTF-8\\\"\"}}],\"type\":\"security_exception\",\"reason\":\"unable to authenticate user [] for action [cluster:admin/xpack/security/saml/authenticate]\",\"header\":{\"WWW-Authenticate\":\"Basic realm=\\\"security\\\" charset=\\\"UTF-8\\\"\"}},\"status\":401}","wwwAuthenticateDirective":"Basic realm=\"security\" charset=\"UTF-8\""}"}
”

some log in Kibana
"
{"type":"error","@timestamp":"2018-07-16T13:44:40Z","tags":["warning","monitoring-ui","kibana-monitoring"],"pid":81410,"level":"error","e rror":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.2.4]: routing [null]]","name" :"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.2.4]: routing [null]] :: {"p ath":"/.kibana/doc/config%3A6.2.4","query":{},"statusCode":503,"response":"{\"error\":{\"root_cause\":[{\"type\":\ "no_shard_available_action_exception\",\"reason\":\"No shard available for [get [.kibana][doc][config:6.2.4]: routing [null]]\\ "}],\"type\":\"no_shard_available_action_exception\",\"reason\":\"No shard available for [get [.kibana][doc][config:6.2.4]: routing [null]]\"},\"status\":503}"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:295:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:254:7)\n at HttpConnector. (/ usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:159:7)\n at IncomingMessage.bound (/usr/share/kibana/node_modul es/elasticsearch/node_modules/lodash/dist/lodash.js:729:21)\n at emitNone (events.js:91:20)\n at IncomingMessage.emit (events.js:18 5:7)\n at endReadableNT (_stream_readable.js:974:12)\n at _combinedTickCallback (internal/process/next_tick.js:80:11)\n at proce ss._tickDomainCallback (internal/process/next_tick.js:128:9)"},"message":"[no_shard_available_action_exception] No shard available for [g et [.kibana][doc][config:6.2.4]: routing [null]]"}
{"type":"log","@timestamp":"2018-07-16T13:44:40Z","tags":["warning","monitoring-ui","kibana-monitoring"],"pid":81410,"message":"Unable to fetch data from kibana_settings collector"}

"
some log in Elasticsearch
"
org.elasticsearch.action.NoShardAvailableActionException: No shard available for [get [.kibana][doc][config:6.2.4]: routing [null]]
at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$AsyncSingleAction.perform(TransportSingleShardAction. java:209) ~[elasticsearch-6.2.4.jar:6.2.4]
at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$AsyncSingleAction.start(TransportSingleShardAction.ja

7

That's a cluster problem that's not specifically security related.
Something looks wrong with your cluster - like you've got unavailable shards.

Check your cluster health and resolve any issues there.

9

HI Tim
Another question
Does Elasticsearch support SP-initiated SSO?
if the answer is yes ,how to config?

10

The SAML Guide configures the Elastic Stack for SP initiated SSO.

11

Thank you Tim
Another question
The the IdP went the cert of our ELK stack, i do not know we should supply the cert in kibana or the cert in elasticsearch?

12

That request is ambiguous - they could want a bunch of different certificates.

Probably they want the SAML encryption and/or signing certificates, in which case you probably should provide them with a full metadata file.

13

server.ssl.certificate: /home/sshuser/cloud/kibana-6.2.4-linux-x86_64/config/server.crt
server.ssl.key: /home/sshuser/cloud/kibana-6.2.4-linux-x86_64/config/server.key
Is this the certificate configured here？
can we generate the cert and key by ourselves(openssl) instead of elasticsearch-certutil tool?

14

I'm happy to help you here, but you need to show some evidence that you are reading the documentation that I link to.
The questions you are asking are answered in those links.

15

HI Tim
i have fixed the issue above ,thank you

and i have another question.
the IDP provide me a cert, this cert is used to validate the saml response.

the response from idp is
<saml:NameID Format="urn:oasis:names:tc:SAML:2.0:nameid-format:transient" NameQualifier="_https://qcwa.honeywell.com/saml2" SPNameQualifier="elasticsearch_demo" >VGBP2LW7Crd1RfJ8D3cJAD0FnBI</saml:NameID>
VGBP2LW7Crd1RfJ8D3cJAD0FnBI
so i think the use account message is encrypted.
my question is where to config the cert for decrypt the message.

16

Hi

the IDP provide me a cert, this cert is used to validate the saml response.

The cert is used to ensure the authenticity and integrity of the SAML Response by validating the Digital Signature of the response. You don't have to configure this explicitly, the certificate should be included in the metadata file you acquired and you mention in your first message. Assuming you have configured Elasticsearch to read that metadata file (see idp.metadata.path in the SAML Guide that Tim has shared with you, you don't need to do anything else.

This is not encrypted. Transient NameIDs are usually opaque random strings that change every time a user logs in to the SAML Identity Provider.

17

Got it
thank you

18

HI kakavas
i faced another problem
[2018-07-19T00:14:16,276][WARN ][o.e.x.s.a.AuthenticationService] [logging-dev03] Authentication to realm saml3 failed - Provided SAML response is not valid for realm saml/saml3 (Caused by ElasticsearchSecurityException[SAML response yXAxKpCT8ZUdHQ8qZREyADXJSB- is for destination null but this realm uses https://sent2-kibana-dev.dsentience.net:443/api/security/v1/saml])

what could be the reason?

19

What version of Elasticsearch are you running?
This issue should be resolved in 6.3.1

20

the version of my Elasticsearch is 6.2.4,
Is there any other solution to this problem other than an updated version?

21

Reconfigured your IdP to send a Destination parameter.
Not all IdPs do that by default, but every one I've seen has the ability to turn it on in some way.