Elasticsearch deployment advise

Elastic Stack Elasticsearch
1

Hello,

I'm in need to plan a new deployment of elasticsearch. Single node, 128GB
ram, for log indexing (amount 50 million records a day)

  1. What's the best heap size for elasticsearch 1.4 (running Oracle java
    7u72) ?
  2. Is there some kind of query throttling technique to stop deep drill
    downs to prevent ES out of memory errors ?

Thanks in advance.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f1c1c29e-9a25-4015-a7e6-6591e9e09118%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

1 - Depends on your use.
2 - Yes there are, see

On 28 November 2014 at 07:17, Denis J. Cirulis denis.cirulis@gmail.com
wrote:

Hello,

I'm in need to plan a new deployment of elasticsearch. Single node, 128GB
ram, for log indexing (amount 50 million records a day)

  1. What's the best heap size for elasticsearch 1.4 (running Oracle java
    7u72) ?
  2. Is there some kind of query throttling technique to stop deep drill
    downs to prevent ES out of memory errors ?

Thanks in advance.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/f1c1c29e-9a25-4015-a7e6-6591e9e09118%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/f1c1c29e-9a25-4015-a7e6-6591e9e09118%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAF3ZnZnDvMNb%2BAB5ZvMgTw5g-ePNgktQDVGjCYdhX9-FcQ7pLg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

20-30Gb per index a day. I've read in setup guide that heap more than 32Gb is useless, that's why I'm asking.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/dfd0a2a4-505d-4496-8b49-2df00a65fa93%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

4

We have 128gb on some nodes and run 30gb heaps. Lucene memory maps files so
the extra memory would be put to good use. The 32gb memory limit comes from
the JVM compressing pointers. It can't compress after 32 and so you see
everything expand in size.
On Nov 27, 2014 4:18 PM, "Denis J. Cirulis" denis.cirulis@gmail.com wrote:

20-30Gb per index a day. I've read in setup guide that heap more than 32Gb
is useless, that's why I'm asking.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/dfd0a2a4-505d-4496-8b49-2df00a65fa93%40googlegroups.com
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPmjWd0JUV%2BL8hi%3D9_PEML%3D4PdnO%2BfMY_MYXG9ANvPW6Qj0KLQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.