How can I create an alert for one-time events?

Sorry for a very late reply.

Yes, I think you understand it correctly. Adding an action that reports to our Jira might be the solution here.