Multitenancy not working as intended with SLO in Kibana

Elastic Observability SLO
1

Kibana/Elasticsearch Version: 8.17.0

Hi, I am setting up a single Elasticsearch instance for observability use case. We deliberately chose not to go with multiple Elasticsearch cluster and chose a centralized cluster for our use case due to day 2 concerns, i.e. keeping clusters versions +- 2 of each other for Cross Cluster Search to work.

Multitenancy in the underlying data (.slo-observability.sli-* and .slo-observability.summary-*) breaks when different tenants on different Kibana spaces creates SLO. All Kibana spaces SLO's aggregated information from Transforms are indexed these same index, if we were to follow SLO access guide, Kibana space A can see Kibana space B's aggregated information such as service.name etc.

.slo-observability.summary-* has a spaceId field that I could use to control document level access. but as for .slo-observability.sli-* there are no such fields. Any advice is greatly appreciated!

On a side note, not sure if this is a bug. When the original user's role that was used to automatically create the Transforms jobs was removed/renamed. The Transforms jobs would all fail as it is still referencing the older roles. Anyway to resolve this? From what i am seeing, the API does not support updating of this roles.

2

Hi @zman,

Space is a Kibana concept only and has limitation unfortunately.

.slo-observability.summary-* has a spaceId field that I could use to control document level access.

The purpose of this spaceId in the summary index is only to filter the SLOs to be displayed on the UI. But anyone with read access on the sli or summary indices can access any documents, regardless of the spaceId value. So you cannot really "control" document level access with it, or I'm missing something? Can you elaborate on how you would achieve that?

As for the Transforms job failing, this is because the initial user's credentials are used to run the transforms.
You might be able to reset the SLO, it should reinstall every resources with the current user's credentials.
POST {/s/spacename}/api/observability/slos/{id}/_reset

3

hi @Kevin_Delemme thanks for the reply.

The purpose of this spaceId in the summary index is only to filter the SLOs to be displayed on the UI. But anyone with read access on the sli or summary indices can access any documents, regardless of the spaceId value. So you cannot really "control" document level

For this. Under Stack Management -> Security -> Roles. Under Index privileges. I added the following.
Indices is *, Privileges is read, view_index_metadata, write, manage, checked Grant read privileges to specific documents with the following Granted documents query

{
  "bool": {
    "should": [
      {
        "terms: {
          "spaceId": [<kibana space id>]
        }
      }
    ],
    "minimum_should_match": 1
  }
}

Added this role to the user and it was working as intended. This user could not see another Kibana space's SLO's and SLI's in a dashboard that we are using. But since .slo-observability.sli-* did not had that field, when visiting the Observability -> SLOs page. All the SLO had no status update as the user did not have access to .slo-observability.sli-*.

Is there a quick workaround to insert spaceId as a field mapping into .slo-observability.sli-*? like how for Datastreams. there is a logs@custom, metrics@custom, traces@custom ingest pipeline that we can add additional fields?

You might be able to reset the SLO, it should reinstall every resources with the current user's credentials.
POST {/s/spacename}/api/observability/slos/{id}/_reset

Let me go and test if this works. But in my opinion this adds on overheads for day 2 when administrating Kibana in a mult-itenanted environment. The administrator could be creating/refactoring roles and we might not know what will break. Can this be abstracted away from the end users? I found this issue coincidentally and am not sure what other features would break because of role change and reassignment to the user.

4

Thanks for detailing the setup of the role.

I'm going to check further tomorrow but using your role setup, I expect to be able to access the SLOs list page just fine since we use the summary index for that view (for most of the fields at least).
The SLO details page might have issues because it uses the more up-to-date data from the sli index though.

We do have an optional pipeline that can be used to add the spaceId, but it is per SLO: slo-${slo.id}@custom. So that might not work well for you.

I might be able to squeeze a bugfix for adding the spaceId in the sli index for 8.18

The administrator could be creating/refactoring roles and we might not know what will break. Can this be abstracted away from the end users? I found this issue coincidentally and am not sure what other features would break because of role change and reassignment to the user.

We use a Kibana internal user with the secondary auth from the user who make the request when creating/managing SLOs . The user's auth is used for checking the permission on the source index mostly. And the internal kibana user is used for avoiding setting permission like manage transforms and manage pipeline cluster privileges to the end users.
I can't recall exactly the reasons why we couldn't use the kibana internal user directly for all operations. I'll try to dig that up also tomorrow.

1 Like
5

Quick question @zman, are you on trial?
I was made aware that this ABAC feature is for Platinum or Enterprise license, so if you're currently in trial, you will lose access to this feature when it ends.

Just want to bring that to your attention.

6

HI @Kevin_Delemme , I am on Enterprise License. Thanks for checking!

7

hi @Kevin_Delemme , any updates on this?

8

Hi, sorry got distracted.

I've created an issue for the spaceId in the SLI data. You can follow it here [SLO] Add spaceId to SLI data · Issue #213464 · elastic/kibana · GitHub

I'll see if someone in the team can take it soon