We are building a graph database in top of ElasticSearch( trueno). Previously we where using ES 2.3 without problem to form a cluster of 20 machines using CentOS:
Icon name: computer-server
Chassis: server
Machine ID: 4624fe6856e646b8ac7b3f737f691f2a
Boot ID: 9f37aa289d484c2d9d98fa64319291c5
Operating System: CentOS Linux 7 (Core)
CPE OS Name: cpe:/o:centos:centos:7
Kernel: Linux 3.10.0-327.el7.x86_64
Architecture: x86-64
We don't have root access to these 20 machines, but since ES 2.3 did not had Bootstrap checks we where able to setup a 20 node cluster easily. Now we are migrating to ES 5.3 to take advantage of the Sliced Scroll feature and read data in parallel in order to speedup graph computation in Spark, but we cannot setup the cluster because we need the 20 machines to communicate between themselves, and ES automatically falls into production mode which enforces Bootstrap checks. Some of them can be solved without root and others strictly needs root(such as file descriptors). Ours scenario is a bit uncommon and we agree Bootstrap checks are a good idea since it prevent disasters, but in our case it is not necessary since Trueno will be used as a storage engine mainly for experiments and analytics in research environments. It may be a good idea to provide a setting to bypass these checks.
I mean that some of the checks can be solved without root(like increasing the Java Virtual Machine Heap before running the process). But those like the file descriptors needs root. I have tried workarounds for increasing file descriptors without root access, but no success so far. I have also tried to use several flags to bypass the checks but are not present anymore in 5.3, ES throws an error of unrecognized option.
Okay, I'm sorry to tell you that you need to get your system administrator to configure these values for you.
Alternatively, you can have the system administrator install the RPM package which sets the max map count on package install, and sets the number of file descriptors for the service.
Jason, no worries, we will contact our system administrator for help. Thanks so much for taking your time to help, I deeply appreciate it! Also wonderful job with ElasticSearch, you guys are heroes.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.