OutOfMemoryError: unable to create new native thread

[Goutham_Pratapa]

Hi all,

I am relatively new to ElasticSearch and I Am running into this issue almost every day in my machine.

The following are details about my hardware.

$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 4126117
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 65536
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 4096
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

Ram is 1007GB

Output of:

curl -XGET " localhost:9200/_nodes/thread_pool?pretty"

is as follows;

"thread_pool" : {
        "watcher" : {
          "type" : "fixed",
          "min" : 80,
          "max" : 80,
          "queue_size" : 1000
        },
        "force_merge" : {
          "type" : "fixed",
          "min" : 1,
          "max" : 1,
          "queue_size" : -1
        },
        "security-token-key" : {
          "type" : "fixed",
          "min" : 1,
          "max" : 1,
          "queue_size" : 1000
        },
        "ml_datafeed" : {
          "type" : "fixed",
          "min" : 20,
          "max" : 20,
          "queue_size" : 200
        },
        "fetch_shard_started" : {
          "type" : "scaling",
          "min" : 1,
          "max" : 160,
          "keep_alive" : "5m",
          "queue_size" : -1
        },
        "listener" : {
          "type" : "fixed",
          "min" : 10,
          "max" : 10,
          "queue_size" : -1
        },

I have increased the Heapsize as well in jvm.options

-Xms31g
-Xmx31g

Can anybody help me in understanding the issue in my machine or my ES setup?

ES-version: 6.8

Thanks in advance