Hi @erikg! I don't think there is a way to achieve this OOTB. I would recommend looking into leveraging a Painless script.
Docs: Watcher condition context | Painless Scripting Language [8.13] | Elastic
This is also a similar post with suggestions: Alert Notification just once