It does seem pretty suspicious that ES is only using 4GB memory. Is that heap utilization or overall memory?
FYI, we've been running ES in production in Docker for about 3 years and have never experienced this type of issue. My first thought was that maybe you had a memory limit set on the container but if you're seeing the full 31GB heap available then that wouldn't be the case. We haven't yet tested 2.4 though so I guess there could be something there.
Kimbro