Suggestion for Server configuration

Elastic Stack Elasticsearch
1

Hi,

Planning for new set of servers for hosting ES and trying to pin on the
configuration. Of course, minimizing cost is an important factor besides
keeping performance steady.

Brief description of data:

20 partitions/blocks (ES type) of data.
Max 500000 documents per block.
50 fields per document.
10 fields have high cardinal values (avg cardinal value for each field =
100)
Other 40 fields are keyword analyzed.

Brief description of operation:

Search using maximum of 10 fields, both normal as well as high cardinal ones
.
Faceting on both normal as well as high cardinal fields, but faceting only
on one type (=~ 500000 docs) in a single query.
5 concurrent user.

Server Configuration options:

Core - 4/8/12/24
Does having more processor has much to do with ES performance? Will basic
quad core serve my purpose?

RAM(GB) - 8/16/32
Having more memory improves faceting, but is there any limit I can consider?

Number of servers -
Is distributing data over multiple servers better?
Or hosting multiple nodes on a single server works well?

Hoping for suggestion from experts.
Thanks,
-- Sujoy.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

Hi Sujoy ,
Your query is absolutely right . I have answer of your one question that
Is distributing data over multiple servers better?
Or hosting multiple nodes on a single server works well?
It is always better to use multiple nodes rather than single server .
Because multiple shards will works parallel with full use of processor. and there will low overhead on single machine.

Thanks ,
Sanjay

3

CPU helps for analyzing queries and indexing, as well as geo functions. ES
is highly concurrent and will make use of all the cores/processors you give
it. So it really comes down to how much CPU power you need for your
situation.

Memory helps faceting, sorting, filters, OS file system cache. Generally
speaking, ES is very memory hungry. It is optimized to trade memory usage
for speed...so when in doubt add more memory. 30GB is the max recommended
heap size without delving into special configurations due to JVM
limitations (so 60GB total, since you want 1/2 to be allocated to OS file
cache). High cardinality facets in particular will eat memory, as will
sorting large datasets.

Disk I/O is important to indexing speed.

Multiple nodes on a single server does not make sense except for a few
select situations. ES will happily utilize all the resources you give it
without needing extra server instances running.

-Zach

On Wednesday, January 30, 2013 7:00:42 AM UTC-5, Sujoy Sett wrote:

Hi,

Planning for new set of servers for hosting ES and trying to pin on the
configuration. Of course, minimizing cost is an important factor besides
keeping performance steady.

Brief description of data:

20 partitions/blocks (ES type) of data.
Max 500000 documents per block.
50 fields per document.
10 fields have high cardinal values (avg cardinal value for each field =
100)
Other 40 fields are keyword analyzed.

Brief description of operation:

Search using maximum of 10 fields, both normal as well as high cardinal
ones.
Faceting on both normal as well as high cardinal fields, but faceting only
on one type (=~ 500000 docs) in a single query.
5 concurrent user.

Server Configuration options:

Core - 4/8/12/24
Does having more processor has much to do with ES performance? Will basic
quad core serve my purpose?

RAM(GB) - 8/16/32
Having more memory improves faceting, but is there any limit I can
consider?

Number of servers -
Is distributing data over multiple servers better?
Or hosting multiple nodes on a single server works well?

Hoping for suggestion from experts.
Thanks,
-- Sujoy.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

4

Thanks to both for the responses.
The suggestions help a lot.

-- Sujoy.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.