We choose all fied found on the documentation in order to have everything and to choose the proper field to work on. It's working fine but we discovered that all important information are stored in json format into the field context alerts and it seems it's not possible to exploit them using stand visualisation.
After some research we found that result are stored into the .siem-signals are properly dispatch into several exploitable fields.
We have also noticed that when using the machine learning solution as the results are stored into index using the json format.
Our solution will to have a logstash to read the information and to push it back using good fields but i think we are missing something here. Can you help us ?
Can you please share what stack version are you on and what are you ultimately trying to achieve?
Detection alerts are Elasticsearch documents that a rule writes to .siem-signals-* or .alerts-security.alerts-* indices (depending on the stack version you use). Every alert is written as a separate document, but the rule might write multiple alerts per run. You shouldn't need to reindex alerts as is to a separate index using an Index Connector -- unless you have specific requirements. It should be possible to build visualizations and dashboards on top of the alerts indices -- just make sure your users have access to them. You can learn more about the RBAC in the docs:
We are actually using the 7.17.9 version. We would like to push theses informations into another index as we plan to add more stuff with other solutions... And we prefer to avoid playing with the system index.
How we can change that ? Any specific options to put into the connector ?
but the form of the index connector doesn't allow to specify non-valid JSON (related issue) and it seems you can't save a rule with a non-valid action body anymore. UPD: you can save a Webhook action with an invalid body (so you can use the workaround), but you can't do the same with an Index action - this one strictly requires the body to be valid JSON (so the workaround won't work).
FWIW one of our teams was planning to work on adding support for triggering rule actions per each generated alert separately, which might be a good option for reindexing them as is into a separate index. I'm not aware of any timeframes though.
And we prefer to avoid playing with the system index.
Unless you need additional or transformed data in your own index (meaning not the alerts as is), you should feel safe and free to read from the system .alerts-security.alerts-* index. This index is managed by the Security app and alerts are written to it by the app itself, you don't need to configure anything to enable that.
Let me know if this information was helpful or not.
Just curious, do you have any public issue tracking this? This is one of the main features that the Kibana Alert is lacking and that is forcing us to stay using a third-party tool, ElastAlert, to trigger our alerts.
@leandrojmp I reached out to my colleagues and found out that this is on the short-term roadmap, but is going to be done as part of another feature called Conditional Actions. There's a public issue for that:
We will need to update our forEach alert feature to filter alert using alert as data when conditional actions are defined. We will still use the in memory alert to build our actions (for example our context and state)
As I was told, this item refers to the ability to trigger actions per each generated alert separately.
Also, there's another public issue for adding support for writing multiple documents via the Index connector: