Cloud active directory auth (Not SAML)

Elastic Stack Elasticsearch
1

Cloud 8.15.3, using SAML auth now, moving from on-prem using Active Directory. SAML works for interactive users, but apparently won't work for things like ingest and scripts. API keys work for scripts, but I get grumbling that they aren't revoked like our on-prem AD users would be if they move or leave. Another issue is the *beats setup, which needs both Elastic and Kibana access, neither API keys nor SAML work for that, all that I've found so far is a native account and again, it's not revoked as part of our AD procedures.

Replacing beats with Fleet managed agents would work, but I've encountered non-technical resistance...

I read that AD realm's aren't supported in the cloud, but is that still the case? Our on-prem AD realm needs to refer to a CA PEM file and I don't think that's possible in the cloud.

Thanks for any ideas.

2

API Keys should work for beats setup.

Yes, that is still the case.

For ingest, do you really want credentials that are revoked according to employee transitions?
Most people want to have ingest processes that are independent of a specific employee (assuming you are ingesting company data - logs, metrics, catalog items, etc - rather than having individuals ingest their own data).

3

The ingest AD creds aren't tied to a person, but a "function". It's more of a non-technical issue here.

I always load the Kibana dashboards at setup, so I thought that needed Kibana creds. API keys might solve the issues.

I'm testing setup with an API key, but keep getting invalid credentials.

I have a python script that uses the api key that works, so I know the key is valid. The python client just uses the key, beats config needs the different format of key_id:key. I get the key_id from devtools (the "id" field?).

Sanitized error message is below. I tried with an API key that was invalidated and got a different error, saying it was invalidated.

I'm stumped as to why I can't the the api key format correct.

Thanks for the help.

Blockquote
401 Unauthorized: {"error":{"root_cause":[{"type":"security_exception","reason":"unable to authenticate with provided credentials and anonymous access is not allowed for this request","additional_unsuccessful_credentials":"API key: invalid credentials for API key [xxxx]","header":{"WWW-Authenticate":["Basic realm=\"security\", charset=\"UTF-8\"","Bearer realm=\"security\"","ApiKey"]}}],"type":"security_exception","reason":"unable to authenticate with provided credentials and anonymous access is not allowed for this request","additional_unsuccessful_credentials":"API key: invalid credentials for API key [xxxx]","header":{"WWW-Authenticate":["Basic realm=\"security\", charset=\"UTF-8\"","Bearer realm=\"security\"","ApiKey"]}},"status":401}","service.name":"filebeat","ecs.version":"1.6.0"}

4

Ah, when you create an API key with Kibana, it returns the combined id:key encoded. More testing needed, after I figure out how to get the id:key decoded...