This is purely an educational question. We hit the well known I/O timeout problem due to the following setup of timeouts:
Agent | Load Balancer | APM server 10s | 60s | 30s
After changing to
10s – 15s – 30s everything works as expected.
I tried to figure out why it doesn't work in the first place, but neither my limited networking knowledge, neither googling and source reading helped me. I would appreciate it if you could explain why this is happening?