Hi! Thanks for writing in. Let me see if I can help. May I ask what version you're on?
I was wondering if your concerns were regarding the actual data still being available in the index, which would relate to ILM or Index LIfecycle Management, with relating docs here:
But, if the problem exists more within the Alert creation, can I confirm if you're using Kibana Alerting to achieve this? If so, which Rule type are you using? Knowing that we can guide you through the creation. Some relating Kibana Alerting docs are linked below, and I hope are helpful:
If you're using a separate system for the alert I'm not sure I'll know how to help within that context. If dev-tools worked, I'm unsure where to go from here, if you can expand on it we can try to help.