# Is the input speed of the elasticsearch cluster linear?
**URL:** https://discuss.elastic.co/t/is-the-input-speed-of-the-elasticsearch-cluster-linear/24464
**Category:** Elasticsearch
**Created:** [June 26, 2015, 9:33pm UTC](https://discuss.elastic.co/t/is-the-input-speed-of-the-elasticsearch-cluster-linear/24464 "2015-06-26T21:33:01Z")
**Posts on this page:** 10
**Page:** 1
### Author:  [@nathaniel](https://discuss.elastic.co/u/nathaniel)
#### Post date: [June 26, 2015, 9:33pm UTC](https://discuss.elastic.co/t/is-the-input-speed-of-the-elasticsearch-cluster-linear/24464/1 "2015-06-26T21:33:01Z")
Hello guys, I am a newbie in elasticsearch. Recently I set up a elasticsearch cluster to save my logs.The cluster has 3 nodes, all of them serves as both master node and data node. I used logstash to foward data to the AWS load balancer, which will then seperate load to the 3 nodes.
The problem is, the throughput is really low, besides, I realized that it is not linear. With one node in the cluster, the throughput is 3666 events /sec when inputing. With two nodes, the throughput is 2750 per node per sec. When I am using all three nodes, the throughput is only 1666 per node per second.
Is the performance supposed to be like this? Or should the throughput linearly increase when adding nodes into the cluster? Hope you guys can give me some help. Thanks in advance.
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [June 26, 2015, 10:52pm UTC](https://discuss.elastic.co/t/is-the-input-speed-of-the-elasticsearch-cluster-linear/24464/2 "2015-06-26T22:52:10Z")
It should increase by adding more nodes.
What does your LS output config look like? What instance size are you using?
---
### Author:  [@nathaniel](https://discuss.elastic.co/u/nathaniel)
#### Post date: [June 29, 2015, 5:00pm UTC](https://discuss.elastic.co/t/is-the-input-speed-of-the-elasticsearch-cluster-linear/24464/3 "2015-06-29T17:00:01Z")
Thanks for answering workolm.
My current situation is that I need to process at lease 34000 events per second, and I am trying to estimate how many notes do I need to process in this speed. I noticed that the processing speed is not linear as the number of node increases, so I wonder how to measure the exact node number.
Here is my LS ouput:
output {
elasticsearch {
bind\_host =\> "localhost"
cluster =\> "elasticsearch-prodII"
workers =\> 8
protocol =\> "http"
flush\_size =\> 200
}
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [June 29, 2015, 5:20pm UTC](https://discuss.elastic.co/t/is-the-input-speed-of-the-elasticsearch-cluster-linear/24464/4 "2015-06-29T17:20:01Z")
If you have the standard 1 replica configured, I would expect 2 nodes to perform about the same as a single node as both nodes would hold and index all data. Once you add nodes from there on, you should see improved throughput assuming you do not change the number of replicas.
This also assumes that you are sending traffic to all nodes and that the nodes are not deployed on a single node where they fight for the same resources.
Your output configuration seems to suggest that you are sending all load to a single node. Is that the case? What type of hardware configuration are you using?
---
### Author:  [@nathaniel](https://discuss.elastic.co/u/nathaniel)
#### Post date: [June 29, 2015, 5:39pm UTC](https://discuss.elastic.co/t/is-the-input-speed-of-the-elasticsearch-cluster-linear/24464/5 "2015-06-29T17:39:07Z")
Hello Christian, thanks so much for answering.
My cluster structure is that I have three nodes, each running a redis server, a logstash and an elasticsearch, and the three nodes are in the same elasticsearch cluster. These three nodes all act as both data node and master node. All the data input comes from an AWS load balancer, which seperates the load equally to the three nodes' redis server. The logstash reads the data from the redis server, and then send to its local elasticsearch. This is why my LS output is sending to localhost.
To wrap it up, I'm using 3 nodes as a cluster, each running logstasn and elasticsearch. The logstash send the data to the local elasticsearch.
I am trully a newbie, I should learn more about this. Do you think this structure works?
As for the hardware configuration, I am using AWS m4.xlarge node. Which has 4 core, 16G RAM. They claim the m4 nodes are using intel Xenon processor, but after all, they are VMs, so don't raise your expectaton 😄
Thanks for answering !
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [June 29, 2015, 6:07pm UTC](https://discuss.elastic.co/t/is-the-input-speed-of-the-elasticsearch-cluster-linear/24464/6 "2015-06-29T18:07:22Z")
I would recommend monitoring the servers during indexing and trying to identify what is the bottleneck and limiting performance, e.g. CPU, disk I/O, memory pressure. As both Logstash and indexing in Elasticsearch can be CPU intensive, that would be what I check first.
---
### Author:  [@nathaniel](https://discuss.elastic.co/u/nathaniel)
#### Post date: [June 29, 2015, 6:09pm UTC](https://discuss.elastic.co/t/is-the-input-speed-of-the-elasticsearch-cluster-linear/24464/7 "2015-06-29T18:09:54Z")
Great advice. Thanks so much Christian.
I noticed that the cpu is the bound. It is nearly fully occupied the whole time.
---
### Author:  [@nik9000](https://discuss.elastic.co/u/nik9000)
#### Post date: [June 29, 2015, 6:24pm UTC](https://discuss.elastic.co/t/is-the-input-speed-of-the-elasticsearch-cluster-linear/24464/8 "2015-06-29T18:24:19Z")
The hot\_threads will be useful there, I think.
Usually the right way to get optimal performance is to have one copy of the shard per server - so if you have 6 servers you want your index in 3 shards with 1 replica for 6 ( (1+1)\*3 ) total indexes. There are lots of parameters you can tweak to sacrifice query performance for indexing performance. Have a look at "update index settings". hot\_threads will probably lead you somewhere though.
---
### Author:  [@nathaniel](https://discuss.elastic.co/u/nathaniel)
#### Post date: [June 29, 2015, 6:28pm UTC](https://discuss.elastic.co/t/is-the-input-speed-of-the-elasticsearch-cluster-linear/24464/9 "2015-06-29T18:28:11Z")
Thanks so much Nik! You pointed out a new direction for me. I'll go check it out.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 12:04am UTC](https://discuss.elastic.co/t/is-the-input-speed-of-the-elasticsearch-cluster-linear/24464/10 "2017-07-06T00:04:43Z")