There are many reasons that may cause this, just to name a few
- benchmarking tool setup ( do they show correct numbers?)
- network bandwidth limits
- cluster setup (e.g. complex mapping, high latency between nodes)
- pattern of the data input
- method of data input (bulk vs. index, HTTP vs. Java API)
- concurrency of server/client connections
- thread pooling resource limits
- SSD setup (e.g. RAID 0)
- resource limits due to hardware setup (host OS often put hard
restrictions on VM guest OS resources)
- etc.
First I recommend testing a single VM for basic performance, especially SSD
raw write throughput.
Then, testing document processing, without and with a single node ES, for
measuring source processing overhead on client side, to eliminate issues
cause by clients.
Then, indexing to single node, and repeating this by doubling nodes each
step, for measuring the performance gain.
Do you see suspicious measurements even on a single node?
Also noteworthy is that VMs are generally quite slowish, compared to bare
metal. Performance depends on VM setup of the host OS. But do not expect
miracles.
There are many ES monitoring tools/plugins that can assist you in getting
metrics.
Jörg
On Mon, Jun 9, 2014 at 5:40 PM, pranav amin parulpatel25@gmail.com wrote:
Hi all,
While doing some prototyping in ES using SSD's we got some good Write TPS.
But the Write TPS saturated after adding some more nodes!
Here are the details i used for prototyping -
Requirement: To read data as soon as possible since the read is followed
by write.
Version of ES:1.0.0
Document Size:144 KB
Use of SSD for Storage: Yes
Benchmarking Tool: Soap UI or Jmeter
VM: Ubuntu, 64 Bit OS
Total Nodes: 12
Total Shards: 60
Threads: 200
Replica: 2
Index Shards: 20
Total Index:1
Hardware configuration: 4 CPU, 6 GB RAM, 3 GB Heap
Using the above setup we got Write TPS ~= 500.
We wanted to know by adding more node if we can increase our Write TPS.
But we couldn't.
- By adding 3 more nodes (i..e Total Nodes = 15) the TPS just increase by
10 i.e. ~= 510.
- Adding more Hardware like CPU, RAM and increasing Heap didn't help as
well [8 CPU, 12 GB RAM, 5 GB Heap].
Can someone help out or point ideas what will be wrong? Conceptually ES
should scale in terms of Write & Read TPS by adding more nodes. However we
aren't able to get that.
Much appreciated if someone can point us in the right direction. Let me
know if more information is needed.
Thanks
Pranav.
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1e34d7c7-d3da-40c7-8fca-16281494065b%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/1e34d7c7-d3da-40c7-8fca-16281494065b%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHnCgkUE0ZG9eprGTPzipP_H2wd7yCRX_go0ukZfW_o6w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.