Adding a low configuration server as an additional node to elasticsearch cluster?

Elastic Stack Elasticsearch
1

Hi ,

I have a single node elasticsearch cluster that may be having slow indexing performance (we have logstash's batch size about 2000 and workers at 40 pushing data to elasticsearch single node), so we were thinking to add an additional node to the cluster. However the configuration given to us is 3 nodes with one 4GB and two 8GB RAM servers . would this be enough to improve performance ?

The current single node server has about 32GB RAM and does adding nodes with lower RAM affect how elasticsearch performs ?

2

Index performance is mostly related to disk speed, what is the kind of the disk of your single node?

3

the node is a VM , the disks are HDD ... I assume SSDs are much better in terms of performance . But since its a HDD, are there any recommendations ?


root@prod-nexus-app03:~# lsblk -d -o name,rota
NAME ROTA
sda     1
sdb     1
sr0     1
root@prod-nexus-app03:~#

Speeds 

root@prod-nexus-app03:~# dd if=/dev/zero of=/tmp/output bs=8k count=10k; rm -f /tmp/output
10240+0 records in
10240+0 records out
83886080 bytes (84 MB, 80 MiB) copied, 0.0923411 s, 908 MB/s
4

HDDs are pretty slow, the recommendation is to use SSDs.

There is not much you can do to try to improve the indexing speed besides changing the disk, but first read this documentation.

You can try to reduce the number of replicas if you are using it and increase the refresh_interval setting of your indices.

I don't see any way where adding other nodes would help in this case, on the contrary, it can impact even more on the performance.

1 Like
5

the replica is already set to zero, and the refresh interval is set at the default 1s. The VMs will be provisioned soon, i'll test it out and see if it works in our use case . Ours is not a data-stream, but will be an on-demand setup scenario where,

User will chose an IP and a predefined set of logs will be fetched and parsed by logstash -> indexed in elasticsearch and populated in kibana dashboards

6

Try to increase it to 30s for example, using refresh interval of 1s with an HDD disk can really impact the performance, increasing it can help a little.

2 Likes
7

will do. Thank you for the help :slight_smile:

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.