# Multiple Elasticsearch Nodes on One Host
**URL:** https://discuss.elastic.co/t/multiple-elasticsearch-nodes-on-one-host/82771
**Category:** Elasticsearch
**Created:** [April 18, 2017, 8:10pm UTC](https://discuss.elastic.co/t/multiple-elasticsearch-nodes-on-one-host/82771 "2017-04-18T20:10:47Z")
**Posts on this page:** 3
**Page:** 1
### Author:  [@seth.yes](https://discuss.elastic.co/u/seth.yes)
#### Post date: [April 18, 2017, 8:10pm UTC](https://discuss.elastic.co/t/multiple-elasticsearch-nodes-on-one-host/82771/1 "2017-04-18T20:10:47Z")
On ES 5.3, Ubuntu 14.04 I have a few servers with both HDD and SSD drives mapped to two different EXT4 filesystems:
`/elasticsearch/warm # HDD /elasticsearch/hot # SSD`
In order to increase my indexing capacity, I want to start two different ES nodes on each of these servers.
I couldn't really find much helpful documentation on doing this so I gave it a shot with what I know about ES but I've been running into issues. I navigated to the ES home (/usr/share/elasticsearch) and ran the command
```auto
sudo -u elasticsearch ./bin/elasticsearch
```
The primary instance of ES (using the HDD) has configuration at /etc/elasticsearch
This secondary instance ES (Using the SSD) has configuration at /usr/share/elasticsearch/config
#### Primary elasticsearch.yml:
```auto
cluster.name: elastic-cluster1
node.name: warm-node
# DATA NODE ROLE
node.master: false
node.data: true
node.ingest: false
path.data: /elasticsearch/warm/data/ #HDD
path.logs: /elasticsearch/warm/logs/
bootstrap.memory_lock: true
network.host: 10.191.5.54
network.bind_host: 0.0.0.0
http.port: 9200
discovery.zen.ping.unicast.hosts: ["legitimate-hostnames-in-my-cluster"]
discovery.zen.minimum_master_nodes: 3
gateway.recover_after_nodes: 5
xpack.security.enabled: false
```
#### Secondary elasticsearch.yml:
```auto
cluster.name: elastic-cluster1
node.name: hot-node
# DATA NODE ROLE
node.master: false
node.data: true
node.ingest: false
path.data: /elasticsearch/hot/data/ #SSD
path.logs: /elasticsearch/warm/logs/hot/ #hot logs to HDD
bootstrap.memory_lock: true
network.bind_host: 0.0.0.0
http.port: 9202
transport.tcp.port: 9302
discovery.zen.ping.unicast.hosts: ["legitimate-hostnames-in-my-cluster"]
discovery.zen.minimum_master_nodes: 3
gateway.recover_after_nodes: 5
xpack.security.enabled: false
```
The first node running on the HDD is working fine, however when I attempt to startup the secondary node by issuing the command:
```auto
sudo -u elasticsearch ./bin/elasticsearch -Epath.conf=/usr/share/elasticsearch/config/
```
I receive the error:
```auto
[2017-04-18T14:09:12,482][INFO][o.e.n.Node] [hot-node] initializing ...
[2017-04-18T14:09:12,576][INFO][o.e.e.NodeEnvironment] [hot-node] using [1] data paths, mounts [[/elasticsearch/hot (/dev/mapper/elasticindex-elastichot)]], net usable_space [492.4gb], net total_space [518.9gb], spins? [no], types [ext4]
[2017-04-18T14:09:12,577][INFO][o.e.e.NodeEnvironment] [hot-node] heap size [1.9gb], compressed ordinary object pointers [true]
[2017-04-18T14:09:12,578][INFO][o.e.n.Node] [hot-node] node name [hot-node], node ID [x-ghbSl5Rf6ObX50KRY4DQ]
[2017-04-18T14:09:12,579][INFO][o.e.n.Node] [hot-node] version[5.3.0], pid[81161], build[3adb13b/2017-03-23T03:31:50.652Z], OS[Linux/4.4.0-57-generic/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_121/25.121-b13]
[2017-04-18T14:09:13,988][INFO][o.e.p.PluginsService] [hot-node] loaded module [aggs-matrix-stats]
[2017-04-18T14:09:13,989][INFO][o.e.p.PluginsService] [hot-node] loaded module [ingest-common]
[2017-04-18T14:09:13,989][INFO][o.e.p.PluginsService] [hot-node] loaded module [lang-expression]
[2017-04-18T14:09:13,989][INFO][o.e.p.PluginsService] [hot-node] loaded module [lang-groovy]
[2017-04-18T14:09:13,989][INFO][o.e.p.PluginsService] [hot-node] loaded module [lang-mustache]
[2017-04-18T14:09:13,989][INFO][o.e.p.PluginsService] [hot-node] loaded module [lang-painless]
[2017-04-18T14:09:13,989][INFO][o.e.p.PluginsService] [hot-node] loaded module [percolator]
[2017-04-18T14:09:13,990][INFO][o.e.p.PluginsService] [hot-node] loaded module [reindex]
[2017-04-18T14:09:13,990][INFO][o.e.p.PluginsService] [hot-node] loaded module [transport-netty3]
[2017-04-18T14:09:13,990][INFO][o.e.p.PluginsService] [hot-node] loaded module [transport-netty4]
[2017-04-18T14:09:13,990][INFO][o.e.p.PluginsService] [hot-node] loaded plugin [x-pack]
[2017-04-18T14:09:16,486][INFO][o.e.n.Node] [hot-node] initialized
[2017-04-18T14:09:16,486][INFO][o.e.n.Node] [hot-node] starting ...
[2017-04-18T14:09:16,805][INFO][o.e.t.TransportService] [hot-node] publish_address {127.0.0.1:9302}, bound_addresses {[::]:9302}
[2017-04-18T14:09:16,816][INFO][o.e.b.BootstrapChecks] [hot-node] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
ERROR: bootstrap checks failed
max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]
[2017-04-18T14:09:16,826][INFO][o.e.n.Node] [hot-node] stopping ...
[2017-04-18T14:09:16,875][INFO][o.e.n.Node] [hot-node] stopped
[2017-04-18T14:09:16,875][INFO][o.e.n.Node] [hot-node] closing ...
[2017-04-18T14:09:16,945][INFO][o.e.n.Node] [hot-node] closed
```
So I can see it's using the proper elasticsearch.yml (due to filepaths and ports attempted to be opened), however none of the limits data is being utilized.
I've modified /usr/lib/systemd/system/elasticsearch.service to include:
`LimitNOFILE=65536 LimitMEMLOCK=infinity`
and /etc/default/elasticsearch to include:
```auto
MAX_OPEN_FILES=65536
MAX_LOCKED_MEMORY=unlimited
```
and /etc/security/limits.conf:
```auto
elasticsearch soft memlock unlimited
elasticsearch hard memlock unlimited
elasticsearch - nofile 65536
```
and my jvm.options:
```
-Xms2g
-Xmx2g
```
of which there _should_ be enough free memory:
```
total used free shared buffers cached
Mem: 94G 85G 9.0G 2.1M 223M 75G
```
However I'm still seeing the aforementioned error on startup:
```auto
ERROR: bootstrap checks failed
max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]
```
### Does anyone know what I'm doing wrong or can you point me to a good resource for starting two nodes on one server?
---
### Author:  [@seth.yes](https://discuss.elastic.co/u/seth.yes)
#### Post date: [April 18, 2017, 9:03pm UTC](https://discuss.elastic.co/t/multiple-elasticsearch-nodes-on-one-host/82771/2 "2017-04-18T21:03:39Z")
I figured out how to get it running. I followed a post found [here](https://underyx.me/2015/05/18/raising-the-maximum-number-of-file-descriptors) (excerpt below):
> The actual way to raise your descriptors consists of editing three files:
> /etc/security/limits.conf needs to have these lines in it:
> - soft nofile 64000
> - hard nofile 64000
> root soft nofile 64000
> root hard nofile 64000
> The asterisk at the beginning of the first two lines means 'apply this rule to all users except root’, and you can probably guess that the last two lines set the limit only for the root user. The number at the end is of course, the new limit you’re setting. 64000 is a pretty safe number to use
> /etc/pam.d/common-session needs to have this line in it:
> session required pam\_limits.so
> /etc/pam.d/common-session-noninteractive also needs to have this line in it:
> session required pam\_limits.so
As well as adding to /etc/security/limits.conf:
```auto
elasticsearch hard nofile 65536
elasticsearch soft nofile 65536
```
Then restart the node.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 16, 2017, 9:11pm UTC](https://discuss.elastic.co/t/multiple-elasticsearch-nodes-on-one-host/82771/3 "2017-05-16T21:11:57Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.