Elasticsearch also uses a mix of NioFS and MMapFS for the various files. Ensure that you configure the maximum map count so that there is ample virtual memory available for mmapped files. This can be set temporarily:
sysctl -w vm.max_map_count=262144
Or you can set it permanently by modifying vm.max_map_count setting in your /etc/sysctl.conf.
Assuming the machine has sufficient memory (256GB total, 4 ES nodes w/ 30G each), would it be a fair assumption that 262144 * 4 (1048576) would be a decent value to use here?
I know this setting itself doesn't actually imply any memory usage but rather ES's ability to allocate more mmap areas.
Input is welcomed.
Thanks!