Question on classifying data in an index

I have data getting logged into the "functionbeat-*" index. This is great but there are multiple classifications of data getting put into this index (e.g. gold data, silver data, bronze data).

I essentially have to control what classifications of data users are able to read. So for example, the user "Yoda" can view all classifications (gold, silver, bronze). The user "Anakin" can only view "bronze data".

Is there a way to force multiple indices from a single functionbeat?

Is there a way to create categorization within a single index?

What is the best approach here?

We do offer field and document level security, which would allow you to filter to role based on classifications for documents in a single index. Otherwise you can specify varying output indices based on field values - https://www.elastic.co/guide/en/beats/functionbeat/current/elasticsearch-output.html#index-option-es

These do imply you have a way of adding those classification values in though, and if the function cannot do it, then you could look at a custom ingest pipeline to add it based on some other information in the output.

I think I was missing a critical detail here. Correct me if i'm wrong, but document security is achieve by specifying a query on a index or index pattern? I kept thinking I had to add a special attribute in my data for the indicing to pick up on but i guess that isn't the case.

Can I create a role that does both document level and field level classification? For example, I might have "gold" documents that have fields I don't want to show and fields I do. Can you point me to an example of what that role might look like?

It does rely on having some way of categorising the data. That'd be easiest if there was a key:value you could point to, but you can also do a general text query using the Elasticsearch DSL.

https://www.elastic.co/guide/en/elasticsearch/reference/current/document-level-security.html has some examples.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.