Is SSD better for ES?

Elastic Stack Elasticsearch
1

Hello,
I have my cluster working on servers with enough memory to keep whole index
in it. Also I'm using RAID 10 for hard drives. But I still see that wa (top
command on linux) number is more than 0%, and sometimes it is around 1%. I
need my cluster optimized for performance. And I've tried to use SSD for
one of my server in the cluster. And I see wa==0 most of the time.
I'm thinking to switch all my servers to SSD, but this is expensive, and
before I'll do this, I wanted to ask for an advice:
I thought that once my index is cached in memory by linux all read/writes
to harddrives would be asynchronous, and it doesn't really matter if it is
slower. So, switching to SSD from traditional hard drives, would it do any
good?
Thanks in advance, any ideas and comments are appreciated.
Eugene

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

Hello,

You can have a look at this article:

Regards,

Nicolas

On Sun, Aug 11, 2013 at 4:38 PM, Eugene Strokin eugene@strokin.info wrote:

Hello,
I have my cluster working on servers with enough memory to keep whole
index in it. Also I'm using RAID 10 for hard drives. But I still see that
wa (top command on linux) number is more than 0%, and sometimes it is
around 1%. I need my cluster optimized for performance. And I've tried to
use SSD for one of my server in the cluster. And I see wa==0 most of the
time.
I'm thinking to switch all my servers to SSD, but this is expensive, and
before I'll do this, I wanted to ask for an advice:
I thought that once my index is cached in memory by linux all read/writes
to harddrives would be asynchronous, and it doesn't really matter if it is
slower. So, switching to SSD from traditional hard drives, would it do any
good?
Thanks in advance, any ideas and comments are appreciated.
Eugene

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

Before you switch to SSD on all servers, why not switch just one test
server and measure the difference? Only your personal experience counts.
Do not trust numbers you can't verify for yourself with your very own
data, workload, and toolset.

The "wa" line in top is completely ok if there are only spikes below
10-15 %. It signals the disk I/O system is working as expected, without
irregular delays. With peaks higher than 20% and many processes going
into "D" state, the I/O system is not optimal for the workload.

One misconception with async I/O: there are no asynchronous I/O
operations exposed in the Java 6 API. If Lucene/ES once will move to
Java 7 there may be a future advantage in using the extra async I/O
calls. Note that async I/O is not necessarily faster, it is just
spawning operations on separate threads in the background so your I/O
code does no longer need a separate custom thread handling to wait for
completion. For example, async I/O does not help in burst mode. By
performing heavy asynchronous communication and thread pooling on a
node, ES is already operating very asynchronously.

Some points to consider:

  • SSDs are strong in high IOPS. That is, you can put much more I/O calls
    on an SSD than on a spindle disk. This particularly helps for mixed
    read/write workloads like they occur during Lucene index merging, if you
    change your index often.

  • SSD does not make searches faster, but it can speed the Lucene index
    file loading into memory significantly

  • SSD does not make indexing faster (if you can ramp up a HD RAID 0
    which is fast enough to cope with your Lucene index writing throughput,
    there is only a small difference between HD and SSD)

  • SSD reduces the I/O load and I/O waits, it helps to exploit full CPU
    power because threads are no longer required to wait for long times

  • SSD have no spindles but there are new durability challenges (wear
    leveling, TRIM, etc.). Check your OS, controllers if you are prepared.
    Most up-to-date OS and hardware is prepared for SSD.

  • SSD is not SSD. Check for the differences between manfacturers and models.

If you set up SSD RAID 0, you can get the highest read/write transfer
rate. I recommend SATA3/SAS-2 (6 Gb/s) otherwise your transfer rate
might be cut off. I have a machine with 4x SSD RAID0 and the raw read
rate is 1.5 GB/s and raw writes are 800 MB/s with a SAS 6Gb/s adapter
(app level throughput is much lower) Also check how many ports and how
many PCIe lanes your board have connected to the controller (SSD can hit
PCIe 1x limits if you are interested in cheap controller card solutions).

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2 Likes