From time to time most of my configured HTTP monitors fail with next errors:
read: connection reset by peer
http: request timed out while waiting for response (Client.Timeout exceeded while awaiting headers)
I tried to increase schedule interval for them form 1 minute to 2 or 3, it a bit helped since I started less frequently see those errors, however they occur from time to time.
In Heartbeat logs I do not see any errors. Those errors I see in Kibana Uptime section.
All of those monitors have the next pattern:
- type: http
schedule: '@every 2m'
All of those instances are being resolved fine without delays.
My heartbeat instance is being run in Kubernetes cluster.
What could be the root of issue for these errors?
I use 6.7.2 version of all Elastic stack.