Elasticsearch does not set the read/write in 4k blocks, this is up to the
JVM/OS. Elastisearch uses Lucene and Lucene wraps all reads and write into
Java I/O streams, where Java uses the OS layer, and the OS uses the file
system drivers.
There are some rule of thumb: most important, don't mess with the block
size. Because the virtual memory page size of Linux is 4k, and
reading/writing pages from devices to memory should be fast, you will often
find 4k blocksize and a factor of 4k for buffers of random access workload
pattern. The Linux distributions come with good defaults. There is also
readahead buffering on fs driver layer for fast reads but these values
strongly depend on the filesystem and drive type. You can try if ES loads
Lucene indices files much faster if you increase the readahead buffering of
the fs but I doubt it. Try with tools like dd to get an impression of how
fast your file system works.
Elasticsearch/Lucene always appends data to the index like a log device.
There are no in-place updates. Old files are dropped after being copied to
new files.
Today the main choice is between SSDs and disk spindle drives. SSDs are
preferred because of the high IOPS numbers and bulk transfer rates, this
reduces the IO system load significantly.
Most important setting is RAID and the noatime parameter on fs mount to
disable file access modification time writing. This saves a lot of
unnecessary seeks and writes on spindle drives.
Best performance for Elasticsearch searches is dependent on RAM, not on
disks. The more RAM you have, the more files the OS is caching. To set up
mmapfs and give enough fs cache to Elasticsearch is one of the keys to
search performance.
Jörg
On Fri, Jul 18, 2014 at 12:20 PM, horst knete baduncle23@hotmail.de wrote:
Hey Guys,
we are going to expand our development ES-System to an productive
environment and we got fresh new storage which waits to be formatted.
In order to get the best performance for elasticsearch, we´d like to know
what the recommended settings are.
Does elasticsearch reads/writes in a sequential way or are there much
random access on the indices?
Which block-size would you recommend? ( i´ve heard elasticsearch
reads/writes in 4k blocks, right? )
Thanks for response
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/8181f3db-1ac7-4c32-aece-169687eb538c%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/8181f3db-1ac7-4c32-aece-169687eb538c%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHTr%2BtASS-nXj%3DDfRaV8qSqPnieVMFdjJJjcJc40B-1bQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.