Active alert from a deleted rule

I noticed 2 issues related to Kibana Alerts:
In my Kibana, there are 2 active alerts for the same rule active all the time - even though one of them is from 3 months in the past and should be long recovered.

A second issue is that when I delete a rule, the alert is still present.
There should be a way to dismiss alerts easily, it's not possible at the moment.

Please advise on how to clear false alerts.