Configuration advice with hard numbers?

Elastic Stack Elasticsearch
1

Greetings,

I have found plenty of documents on guiding configuration tuning and plenty
of use cases showing performance for a particular collection, but is there
any
advice which provides a reasonable hard number estimate based upon
collection size ? (i.e. number of documents, total disk size of
documents...)

We plan to have 4 million documents with around a terrabyte of storage in
raw json format.

Thanks for any pointers.....love this product, it is fantastic....although
it is still early in our relationship.

Take care,

-tony

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/4644a4d4-cd33-42ea-b32b-735f2f7b76e4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

No, because there are still a number of variables that factor into this to
make it into another "it depends".

Things like heap, CPU, disk, query types, ES version, cluster and index
size and setup,

On 31 January 2015 at 04:39, Tony Neil captaintneil@gmail.com wrote:

Greetings,

I have found plenty of documents on guiding configuration tuning and
plenty of use cases showing performance for a particular collection, but is
there any
advice which provides a reasonable hard number estimate based upon
collection size ? (i.e. number of documents, total disk size of
documents...)

We plan to have 4 million documents with around a terrabyte of storage in
raw json format.

Thanks for any pointers.....love this product, it is fantastic....although
it is still early in our relationship.

Take care,

-tony

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/4644a4d4-cd33-42ea-b32b-735f2f7b76e4%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/4644a4d4-cd33-42ea-b32b-735f2f7b76e4%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-1TbsCjAYw4ZdG1_UF05h%2BiWvdH1LAkxGm2uss0uDgow%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

You can get poor performance with 1TB of data if:

  • Your queries are not right (e.g. not using right filters/using wrong
    ranges etc)
  • Inferior storage hardware
  • Insufficient memory (RAM)

You can get really great performance on the same data set of 1TB

  • if you have SSDs
  • enough RAM to cache the data
  • right cluster topology (e.g. dedicated data/client/master nodes)

So the answer depends on how a lot of factors. :slight_smile:

On Friday, 30 January 2015 09:39:11 UTC-8, Tony Neil wrote:

Greetings,

I have found plenty of documents on guiding configuration tuning and
plenty of use cases showing performance for a particular collection, but is
there any
advice which provides a reasonable hard number estimate based upon
collection size ? (i.e. number of documents, total disk size of
documents...)

We plan to have 4 million documents with around a terrabyte of storage in
raw json format.

Thanks for any pointers.....love this product, it is fantastic....although
it is still early in our relationship.

Take care,

-tony

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/150f5b69-e047-4307-a844-f252620928fc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.