How to pass the MAX_OPEN_FILES argument to JVM?

Elastic Stack Elasticsearch
1

Hi there,

I'm trying to pass MAX_OPEN_FILES parameter to JVM.

Actually, I am trying to increase fd on mac os x. I made it with launchctl command but the default behavior is to set the limit value by JVM, which is 10240. Therefore, I need pass MaxFDLimit parameter jvm when Elasticsearch starting.

I tried to use shell file

export MAX_OPEN_FILES=1000000

ES_JAVA_OPTS="$ES_JAVA_OPTS -Des.path.conf=$ES_CONF_DIR"

#ES_JAVA_OPTS="$ES_JAVA_OPTS -Des.path.data=$ES_DATA_DIR -Des.path.conf=$ES_CONF_DIR"
export ES_JAVA_OPTS

cd $ES_HOME
exec $ES_HOME/bin/elasticsearch

It did not work.

Does anyone have any suggestions?

Thanks.

2

Doesn't the JVM by default use the process's fd limit (OPEN_MAX), indicating that you're not upping that limit correctly? Now, if you do need to pass -XX:-MaxFDLimit I don't see why setting MAX_OPEN_FILES would help. Wouldn't you actually want to append -XX:-MaxFDLimit to ES_JAVA_OPTS? Seemingly relevant: http://stackoverflow.com/a/23530494/414355

3

Hi @magnusbaeck,

I used launchctl command for increasing fd.

sudo launchctl limit maxfiles 1000000 1000000
$ ulimit -n
1000000

After the above step and restarted Elasticsearch, I'm getting the following response from Elasticsearch when I uses this command: curl -XGET 'localhost:9200/_nodes/process?pretty'

{
  "cluster_name" : "elasticsearch",
  "nodes" : {
    "y92KgXVuSdiB5khExFWA9g" : {
      "name" : "Hyperion",
      "transport_address" : "inet[/192.168.1.21:9300]",
      "host" : "terrasacer-air.local",
      "ip" : "192.168.1.21",
      "version" : "1.7.1",
      "build" : "b88f43f",
      "http_address" : "inet[/192.168.1.21:9200]",
      "process" : {
        "refresh_interval_in_millis" : 1000,
        "id" : 1395,
        "max_file_descriptors" : 10240,
        "mlockall" : false
      }
    }
  }
}

I can use the -XX:-MaxFDLimit in terminal but I can't use it in a shell file.

4

I can use the -XX:-MaxFDLimit in terminal but I can't use it in a shell file.

What do you mean? Please be explicit.

5

For example, when I used sudo bin/elasticsearch -XX:-MaxFDLimit then I'm getting the response:

"max_file_descriptors" : 1000000,

I want to get a similar result when starting Elasticsearch with a shell file.

export ES_HOME=/Users/terrasacer/Downloads/elasticsearch
export ES_CONF_DIR=$ES_HOME/config
export ES_DATA_DIR=$ES_HOME/data
export CLASSPATH=$ES_HOME/lib/elasticsearch-*.jar:$ES_HOME/lib/*:$ES_HOME/lib/sigar/*

ES_JAVA_OPTS="$ES_JAVA_OPTS -Des.path.data=$ES_DATA_DIR -Des.path.conf=$ES_CONF_DIR"

export ES_JAVA_OPTS

cd $ES_HOME
exec $ES_HOME/bin/elasticsearch

How do I use the above file -XX:-MaxFDLimit JVM option?

6

It's hard to believe that adding it to ES_JAVA_OPTS or mimicking what you did in the shell (i.e. passing it directly to bin/elasticsearch) doesn't work. If that indeed is the case, inspecting the actual, full, command line in both cases should provide clues.

7

The -XX:-MaxFDLimit flag for JVM is not doing anything useful. It was meant for old Solaris 8 OS, where the kernel allocates only 256 file descriptors per process by default.

You should be aware that this flag will be removed in future JDK releases.

This flag is not needed any more. In Linux, you have to adjust the kernel settings in /etc/security/limits.conf but the default settings for file descriptors is already increased to 10240 in current distributions, which is more than you will need for Elasticsearch.

8

Gentlemen, thank you for your answers.

@jprante

Is this documented anywhere?