I need some more insight on .siem-security index creation by kibana.
We have created 3 spaces in kibana and enabled the same Security -> elastic rules under all 3 spaces (eg: Whitespace Padding in Process Command Line).
we have also assigned the roles and and users per space to view their own data. This settting is working fine with all integrations we installed by centralized fleet agent . However we noticed that siem-security data is getting written in all spaces i.e.
( Space1 generates the alerts with DataStream.namespace=Space1 under siem-security-Space1-* ,siem-security-Space2-* and siem-security-Space3-* )
Based on my understanding , fleet manager is smart enough to filter data based on namespace from elastic ; but kibana is not.
Is that correct understanding ? Or I am missing some point which is causing kibana to add docs to all siem-security- indexes {spaces where same elastic rules is enabled}.
Hey there @niveditakathal -- thanks for the question and using Elastic Security!
So by default, Detection Rules are just going to look at their configured source indices, and if they find any matches based on their configured query/exceptions, they'll go ahead and create corresponding alerts in the alerts index for the Kibana Space in which they reside. There is no inherent space-filtering of source data at the Detection Rule layer.
That said, this functionality can be added a few different ways:
The preferred way for you is probably going to be leveraging Document Level Security since you're already using custom Users/Roles for each of these spaces. All you need to do here is ensure the Users/Roles that are managing your Detection Rules for a given space only have read access to source data with the same namespace, e.g.:
Once you update the Role, have the user go and disable/enable the rules in their space (this will generate a new API key based on this new Role that the Rule will use to query), and now when the Rule runs it will no longer see any data for those other namespaces, and only create alerts for documents matching the same namespace!
Document Level Security is a subscription feature though, so another way of accomplishing similar results would be by configuring exceptions on the rules in each space such that they ignore data from the other spaces. Since exceptions are exclusory by nature, you would need to add an exception that lists all spaces for which you want to ignore data. So if you have three spaces as mentioned, you could add an exception to all rules in Space1 that ignore data from Space2/3 like in the below image:
That said, this doesn't scale as well and has a high level of maintenance if you're creating and deleting spaces or adding new ingest namespaces. Plus exceptions can't be added in bulk yet so it'd be a bit cumbersome to configure it all if you're not using the API.
Hope this helps @niveditakathal, and I'll be sure to forward along your use case to the product folks so we can better fit your needs. Let me know if this works for you or if you have any additional questions -- cheers!
I am working with basic license and with that I can not use document query.
Also, adding rule exception for every space won't be manageable in long term; hence we are trying to filter the documents based on index pattern.
Understandable @niveditakathal -- another option that might work for you is just modifying the raw rule query to include an AND data_stream.namespace: Space1 filter. This has overhead like the exceptions example, but if you have any automation around how your rules are managed between spaces this might be something easy to implement as part of your rule management lifecycle.
BTW, I've forwarded along this usecase to our product folks, so hopefully in the future we can introduce additional ways of segmenting data access based on data_stream.namespace similar to how fleet integration are configured. Appreciate the feedback here with your use case.
@spong Thanks a lot to discuss above use case with product folks.
For now we have created two roles per client with required index patterns and permissions as a workaround.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
