For some time, I have been successfully running Elasticsearch 2.x-based instances of the Elastic Stack in Docker containers using Docker for Windows. I use bespoke Dockerfiles based on images from sebp/elk (I don't want X-Pack). These Docker containers are strictly for small-scale demonstration purposes. I'm interested in Docker for Windows primarily because the people who perform these demos typically use Windows as their native operating system.
I ran into trouble doing the same thing with Elastic 5.x.
The Elasticsearch service failed to start. When I started Elasticsearch manually, by running the
elasticsearch script from the Docker container shell, the shell responded “Killed”. No log in the container would tell me why. I’m a relative *nix newbie. I was flummoxed. I did some Googling, found some possible solutions, and tried various things.
- Open the Docker for Windows Settings dialog
- Select Advanced
- Move the Memory slider up to 3072 MB
- Click Apply
- Restart the Docker container
- Experienced *nix and Docker users: if you haven’t already rolled your eyes, feel free to roll them now
(Docker Community Edition, Version 17.03.0-ce-win1 (10296))
Discussion and questions
I suspect that Elasticsearch was killed by the OOM killer on the Docker host.
In the context of Docker for Windows, the Docker host is MobyLinuxVM, a virtual machine running under Hyper-V. If I knew how to check the logs on that VM—if that’s even possible (I’m not sure; the research that I’ve done indicates that the Docker for Windows developers don’t want you SSHing into that VM)—then I think I would find some log messages to that effect.
I wondered if Hyper-V might helpfully forward such messages to the Windows Event Viewer, but nope.
I think (please feel free to correct me) the underlying issue—why Elasticsearch 2.x worked for me, but 5.x initially didn’t—is the Elasticsearch default JVM heap size: 1 GB in 2.x, 2 GB in 5.x.
I was frustrated by my inability to open a command shell to the MobyLinuxVM. I’m wondering whether I’ll hit other problems that would be easier to solve if I could do that. Can anyone tell me how to do that?
Can anyone tell me how to define sysctl settings in the MobyLinuxVM? That might come in handy.
(I also use Docker on Linux, where I have direct access to the Docker host.)
I haven’t yet attempted to decrease the default Elasticsearch 5.x JVM heap settings, to see how little I can get away with on these small-scale demo instances.