Hi @willemdh!
This definitely sounds like really useful functionality. I found a few resources that may be of use here and could satisfy the behavior you are looking for.
Have you heard of building block rules? In your stated case, you could mark the rule in your example as a building block rule. This would still create the alerts but hide them from the UI so that they don't create unnecessary noise. Then you could create a rule that searches the alerts index for those with high severity. You can find info on building block rules here.
This other forum post might also be of help.
Let us know if those resources address your use case or if we can be of any further help.
Best,
Yara