# Performance 1 shard / 1 node vs 5 shards / 5 nodes **URL:** \ **Category:** Elasticsearch\ **Created:** [July 2, 2012, 9:48pm UTC](https://discuss.elastic.co/t/performance-1-shard-1-node-vs-5-shards-5-nodes/8289 "2012-07-02T21:48:20Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [July 2, 2012, 9:48pm UTC](https://discuss.elastic.co/t/performance-1-shard-1-node-vs-5-shards-5-nodes/8289/1 "2012-07-02T21:48:20Z")
Heya I'm playing with ES to adjust some parameters for my needs and here is what I can see. I'm running on Ubuntu 64 bits, openJDK1.7.0\_03 with elasticsearch 0.19.4 Here is my scenario: I inject "small documents" (less than 10 fields). I inject 1 000 000 docs. I run one search query (with facet). First test : Only one node and one shard with no replica . Injection time : 155 s . Search time : 972 ms Second test : Five nodes (but on the same hardware) and 5 shards with no replica . Injection time : 444 s . Search time : 346 ms As I can see, search time is better with multiple shards (nodes). That's what I was expected. But indexation time is worst (3 times) with multiple shards (nodes). At least, I was thinking that I would get the same result. What do you think ? Is it because I'm running on the same hardware (file system) ? So I have some IOWaits ? Should I expect better results when running five nodes on different boxes or at least different hard disks ? That's not an issue. I can wait for 8 minutes to inject 1 million docs šŸ˜‰ Thanks David. ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [July 2, 2012, 10:59pm UTC](https://discuss.elastic.co/t/performance-1-shard-1-node-vs-5-shards-5-nodes/8289/2 "2012-07-02T22:59:22Z")
How are you indexing the data? Is it using bulk requests, or one doc at a time? If you are using Java, is it a Node Client or a Transport Client? Also, is it a single indexing process/thread? On Mon, Jul 2, 2012 at 11:48 PM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > Heya\*\*\*\* > > * * * > > * * * > > Iā€™m playing with ES to adjust some parameters for my needs and here is > what I can see.\*\*\*\* > > Iā€™m running on Ubuntu 64 bits, openJDK1.7.0\_03 with elasticsearch 0.19.4\*\* > \*\* > > * * * > > Here is my scenario:\*\*\*\* > > I inject ā€œsmall documentsā€ (less than 10 fields).\*\*\*\* > > I inject 1 000 000 docs.\*\*\*\* > > I run one search query (with facet).\*\*\*\* > > * * * > > _First test : Only one node and one shard with no replica_\*\* > > **Ā· Injection time : 155 s** > > **Ā· Search time : 972 ms** > > * * * > > _Second test :_ Five nodes (but on the same hardware) and 5 shards with > no replica\*\*\*\* > > **Ā· Injection time : 444 s** > > **Ā· Search time : 346 ms** > > * * * > > As I can see, search time is better with multiple shards (nodes). Thatā€™s > what I was expected.\*\*\*\* > > But indexation time is worst (3 times) with multiple shards (nodes). At > least, I was thinking that I would get the same resultā€¦\*\*\*\* > > * * * > > What do you think ?\*\*\*\* > > Is it because Iā€™m running on the same hardware (file system) ? So I have > some IOWaits ?\*\*\*\* > > Should I expect better results when running five nodes on different boxes > or at least different hard disks ?\*\*\*\* > > * * * > > Thatā€™s not an issue. I can wait for 8 minutes to inject 1 million docs šŸ˜‰ > > * * * > > * * * > > Thanks\*\*\*\* > > David.\*\*\*\* ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [July 3, 2012, 5:03am UTC](https://discuss.elastic.co/t/performance-1-shard-1-node-vs-5-shards-5-nodes/8289/3 "2012-07-03T05:03:13Z")
One doc at a time. Using NodeJS in a single Thread. -- David Le 3 juil. 2012 Ć  00:59, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) a Ć©crit : > How are you indexing the data? Is it using bulk requests, or one doc at a time? If you are using Java, is it a Node Client or a Transport Client? Also, is it a single indexing process/thread? > > On Mon, Jul 2, 2012 at 11:48 PM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > Heya > > Iā€™m playing with ES to adjust some parameters for my needs and here is what I can see. > > Iā€™m running on Ubuntu 64 bits, openJDK1.7.0\_03 with elasticsearch 0.19.4 > > Here is my scenario: > > I inject ā€œsmall documentsā€ (less than 10 fields). > > I inject 1 000 000 docs. > > I run one search query (with facet). > > First test : Only one node and one shard with no replica > > Ā· Injection time : 155 s > > Ā· Search time : 972 ms > > Second test : Five nodes (but on the same hardware) and 5 shards with no replica > > Ā· Injection time : 444 s > > Ā· Search time : 346 ms > > As I can see, search time is better with multiple shards (nodes). Thatā€™s what I was expected. > > But indexation time is worst (3 times) with multiple shards (nodes). At least, I was thinking that I would get the same resultā€¦ > > What do you think ? > > Is it because Iā€™m running on the same hardware (file system) ? So I have some IOWaits ? > > Should I expect better results when running five nodes on different boxes or at least different hard disks ? > > Thatā€™s not an issue. I can wait for 8 minutes to inject 1 million docs šŸ˜‰ > > Thanks > > David. ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [July 3, 2012, 9:55am UTC](https://discuss.elastic.co/t/performance-1-shard-1-node-vs-5-shards-5-nodes/8289/4 "2012-07-03T09:55:55Z")
So, what happens is that when you move form 1 node to 5 nodes, you pay the price of network between nodes. With 5 nodes, you hit one node, and it needs to send it over to another node. In a single node case, it does not do the extra network hop. On the other hand, if you had more processes indexing data, 5 nodes would speed the time to index. On Tue, Jul 3, 2012 at 7:03 AM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > One doc at a time. > Using NodeJS in a single Thread. > > -- > David > > Le 3 juil. 2012 Ć  00:59, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) a Ć©crit : > > How are you indexing the data? Is it using bulk requests, or one doc at a > time? If you are using Java, is it a Node Client or a Transport Client? > Also, is it a single indexing process/thread? > > On Mon, Jul 2, 2012 at 11:48 PM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > > > Heya\*\*\*\* > > > > * * * > > > > * * * > > > > Iā€™m playing with ES to adjust some parameters for my needs and here is > > what I can see.\*\*\*\* > > > > Iā€™m running on Ubuntu 64 bits, openJDK1.7.0\_03 with elasticsearch 0.19.4\* > > > > * * * > > > > * * * > > > > Here is my scenario:\*\*\*\* > > > > I inject ā€œsmall documentsā€ (less than 10 fields).\*\*\*\* > > > > I inject 1 000 000 docs.\*\*\*\* > > > > I run one search query (with facet).\*\*\*\* > > > > * * * > > > > _First test : Only one node and one shard with no replica_\*\* > > > > **Ā· Injection time : 155 s** > > > > **Ā· Search time : 972 ms** > > > > * * * > > > > _Second test :_ Five nodes (but on the same hardware) and 5 shards with > > no replica\*\*\*\* > > > > **Ā· Injection time : 444 s** > > > > **Ā· Search time : 346 ms** > > > > * * * > > > > As I can see, search time is better with multiple shards (nodes). Thatā€™s > > what I was expected.\*\*\*\* > > > > But indexation time is worst (3 times) with multiple shards (nodes). At > > least, I was thinking that I would get the same resultā€¦\*\*\*\* > > > > * * * > > > > What do you think ?\*\*\*\* > > > > Is it because Iā€™m running on the same hardware (file system) ? So I have > > some IOWaits ?\*\*\*\* > > > > Should I expect better results when running five nodes on different boxes > > or at least different hard disks ?\*\*\*\* > > > > * * * > > > > Thatā€™s not an issue. I can wait for 8 minutes to inject 1 million docs šŸ˜‰ > > > > * * * > > > > * * * > > > > Thanks\*\*\*\* > > > > David.\*\*\*\* ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [July 3, 2012, 9:56am UTC](https://discuss.elastic.co/t/performance-1-shard-1-node-vs-5-shards-5-nodes/8289/5 "2012-07-03T09:56:23Z")
Additionally, it you use bulk index, then you would spread the load as well from a single client. On Tue, Jul 3, 2012 at 11:55 AM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote: > So, what happens is that when you move form 1 node to 5 nodes, you pay the > price of network between nodes. With 5 nodes, you hit one node, and it > needs to send it over to another node. In a single node case, it does not > do the extra network hop. On the other hand, if you had more processes > indexing data, 5 nodes would speed the time to index. > > On Tue, Jul 3, 2012 at 7:03 AM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > > > One doc at a time. > > Using NodeJS in a single Thread. > > > > -- > > David > > > > Le 3 juil. 2012 Ć  00:59, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) a Ć©crit : > > > > How are you indexing the data? Is it using bulk requests, or one doc at a > > time? If you are using Java, is it a Node Client or a Transport Client? > > Also, is it a single indexing process/thread? > > > > On Mon, Jul 2, 2012 at 11:48 PM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > > > > > Heya\*\*\*\* > > > > > > * * * > > > > > > * * * > > > > > > Iā€™m playing with ES to adjust some parameters for my needs and here is > > > what I can see.\*\*\*\* > > > > > > Iā€™m running on Ubuntu 64 bits, openJDK1.7.0\_03 with elasticsearch 0.19.4 > > > > > > * * * > > > > > > * * * > > > > > > Here is my scenario:\*\*\*\* > > > > > > I inject ā€œsmall documentsā€ (less than 10 fields).\*\*\*\* > > > > > > I inject 1 000 000 docs.\*\*\*\* > > > > > > I run one search query (with facet).\*\*\*\* > > > > > > * * * > > > > > > _First test : Only one node and one shard with no replica_\*\* > > > > > > **Ā· Injection time : 155 s** > > > > > > **Ā· Search time : 972 ms** > > > > > > * * * > > > > > > _Second test :_ Five nodes (but on the same hardware) and 5 shards with > > > no replica\*\*\*\* > > > > > > **Ā· Injection time : 444 s** > > > > > > **Ā· Search time : 346 ms** > > > > > > * * * > > > > > > As I can see, search time is better with multiple shards (nodes). Thatā€™s > > > what I was expected.\*\*\*\* > > > > > > But indexation time is worst (3 times) with multiple shards (nodes). At > > > least, I was thinking that I would get the same resultā€¦\*\*\*\* > > > > > > * * * > > > > > > What do you think ?\*\*\*\* > > > > > > Is it because Iā€™m running on the same hardware (file system) ? So I have > > > some IOWaits ?\*\*\*\* > > > > > > Should I expect better results when running five nodes on different > > > boxes or at least different hard disks ?\*\*\*\* > > > > > > * * * > > > > > > Thatā€™s not an issue. I can wait for 8 minutes to inject 1 million docs > > > ;-)\*\*\*\* > > > > > > * * * > > > > > > Thanks\*\*\*\* > > > > > > David.\*\*\*\* ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:21am UTC](https://discuss.elastic.co/t/performance-1-shard-1-node-vs-5-shards-5-nodes/8289/6 "2017-07-06T03:21:41Z")