ES Master suddenly losing connection to data nodes

[lwintergerst (Luca Wintergerst)]

hm.

you can check out my new post.

I think the solution is either the keepalive time of the tcp connections or a kernel bug.

Random node disconnects - Java.io.IOException: Connection timed out

EDIT 3: Solution TLDR : I found a way to finally solve this. This blog post describes it very well. My linux kernel has a bug with the network scatter/gather functionality. Turning this off solves this issue completely! EDIT 2: First try - this was not the Problem!! the OS was closing connections that were still needed. sysctl -w net.ipv4.tcp_keepalive_time=600 net.ipv4.tcp_keepalive_intvl=60 net.ipv4.tcp_keepalive_probes=3 found at We just recently started a ne…