Hello, my goal is to monitor a specific value from an index in ELK. Concretely I would like to alert if the 95th percentile of my nginx request_time is greater than some threshold. What is the best way to accomplish this?
What I Have Tried
Create a Rule using a Metrics Threshold
In this case I do not see the required field (request_time) from my specific index. I have validated that the index pattern configured in Observability > Logs > Settings
contains request_time
and it is a float and can be aggregated. Do I need to add this index in Observability > Metrics > Settings
for this to work, if so is this the recommended way of doing things?
Create a Watcher
Two issues with a watcher right now:
- It does not have an option for 95th percentile when setting up a new watcher
- The Slack Action requires I set up a user, is there a way to use the Slack Action defined in the Alerting section?
Please let me know what the best way to accomplish this is!
Thanks
Anthony