Document level permissions/security with signed search keys also for Elasticsearch indices

Elastic Search
1

Hi,

I've read the documentation how to leverage Workplace Search document-level permissions in app search engine using a signed search key.

Leverage Workplace Search document-level permissions in your search engine | Enterprise Search documentation [8.8] | Elastic

I tried it and it works. I am wondering whether this also workes for app search engines which reference an Elasticsearch index? Maybe if I add a mapping for the _allow_permissions or _deny_permissions to the Elasticsearch index, so "simulate" a Workplace search index.

Would this be possible?

Or I also read about document level security n Elasticsearch itself: Document level security | Elasticsearch Guide [8.8] | Elastic But how to set the identities to the index? In the sample they mentioned the set_security_processor - but this adds security information only for the Kibana logged in user.

In my case I have some existing Elasticsearch indices, populated by some applications. And I would be able to add extra mappings for users or roles which have access to the documents. But I don't know ho to combine with Enterprise Search, App Search, Workplace Search ... Of course I could develop an UI which sets the user id as a query filter - but I wanted to use some pre-built and integrated security features in the backend and not the frontend.

Best regards

Sebastian

2

Hi @sebastianboelling ,

Great questions! I'll reply to each in turn.

Yes, that will work! We've been working on adding DLS support for our newer (non-Workplace Search) connectors, which take this approach exactly. Check out this guide: Leverage DLS from connectors in App Search | Enterprise Search documentation [8.11] | Elastic

But how to set the identities to the index?

If you're doing your own ingestion, you'll have to do this yourself. I don't know where your identity definitions would come from, or what business logic you would use to describe which identity should see which documents. The key bit though is to be able to generate the Elasticsearch Query Syntax filter for a given search user that can limit what they should be able to search over. Once you have crafted such a filter, you can apply it to a Signed Search Key and then give that key to that user.

I wanted to use some pre-built and integrated security features in the backend and not the frontend.

Workplace Search does this "pre built" because the use case for Workplace Search is that it provides 100% of the search experience. But App Search is a platform, and isn't really intended to be a full-service end-user search application. It's intended to facilitate you building a search application on top of it. So you're going to have to write some code (frontend or backend) to define the relevant filters and create the required keys.

Alternatively, you could use Elasticsearch directly, as you've already read about in the guide you linked, where you can use Elasticsearch Roles and Role Mappings to ensure a given Elasticsearch user has a filtered view of a given index. Even with this though, you'd need to create those roles in advance, or would need to add an Elasticsearch Plugin where you define your own CustomRoleProvider.

3

Hi @Sean_Story,

Great answers! I've some new. With 8.9.0 Elastic releases a new approach for implementing DLS with Search Applications.

Document level security | Enterprise Search documentation [8.9] | Elastic

The idea to use access control documents in an "access control index" which is attached to the content documents in an "contect index" is quite interesting. It is quite similar to the contept of external identities of Workplace Search.

But it seems that every content index needs his own access control index, right?

And both are reverenced by their name (.search-acl-filter-indexblah and search-indexblah). Would'nt it be easier to reference them by an field which is either included in the content index or the access control index (or maybe both)?

Then one acces control index could be used for more then one content index - which would be a great feature if you have always the same users for every datasource.

Or why not bind to users to LDAP, AD, SAML etc. ?

For understanding: It is really hard to manage a huge numbers of users for each access control index if you have a lot of datasources and you have always the same users (if you have an central IAM service in your enterprise)..

BTW. Is the name of the access control index .search-acl-filter- as described here How DLS works | Enterprise Search documentation [8.9] | Elastic or without a prefix dot "." as shown here Leverage document-level security from connectors in Search Applications | Enterprise Search documentation [8.9] | Elastic for the SharePoint sample search-acl-filter-sharepoint.

Many thanks,

Sebastian

4

Hi @sebastianboelling ,

it seems that every content index needs his own access control index, right?

Right

And both are reverenced by their name (.search-acl-filter-indexblah and search-indexblah). Would'nt it be easier to reference them by an field which is either included in the content index or the access control index (or maybe both)?

That would be an option, but we chose using index names to correlate them so that permissions couldn't be accidentally or maliciously disassociated. If I had my data in index search-indexblah and permissions in other-acl-index, and was associating them with some mapping in the search-indexblah/_mapping, a change to that mapping for a data migration might remove or alter the association my data security relied on.

Then one acces control index could be used for more then one content index - which would be a great feature if you have always the same users for every datasource.

That is true.

Or why not bind to users to LDAP, AD, SAML etc. ?

Not all customers use LDAP, AD, or SAML. And even if all customers did, not all customers would use their SSO systems for identity management for all their sources of data.

In the long run, we'd like to ensure that if you use an LDAP, AD, or SAML realm for Elasticsearch Authentication we can help you also ensure that your authenticated users are assigned auto-generated Elasticsearch Roles that limit their access to connector-powered indices using DLS. This functionality does not exist at the time of writing this, but stay tuned to our release notes in future versions.

Even once we do have this functionality though, we will not rely solely on it, because we want to adopt a security posture that is accessible to all customers, not just ones who use certain identity management tools.

For understanding: It is really hard to manage a huge numbers of users for each access control index if you have a lot of datasources and you have always the same users (if you have an central IAM service in your enterprise)..

I understand the difficulty, and I empathize. We're taking steps to make this process easier. For example, you may have noticed that while in Workplace Search, External Identities had to be created one at a time, via API, to map users to their identities. And this had to be done before a Permissions Sync, to inform the sync on what user information should be retrieved. In the newer connectors, however, all user identities are retrieved and synced, so that they can be looked up at query time. The Elasticsearch Filter is also transparently available, to put more power in the hands of the developer.

BTW. Is the name of the access control index .search-acl-filter- as described here How DLS works | Enterprise Search documentation [8.9] | Elastic or without a prefix dot "." as shown here Leverage document-level security from connectors in Search Applications | Enterprise Search documentation [8.9] | Elastic for the SharePoint sample search-acl-filter-sharepoint .

with the . is correct. Thank you for flagging the mistake in our docs, I've raised a PR and should have the . added correctly soon.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.