Elasticsearch performance improvment

Elastic Stack Elasticsearch
1

Hi all,

I have 4 nodes, memory size respe respectively 64GB, 64GB, 64GB and 128 GB
for handing 8,577,271,728 Documents and 68 TB data

I open that API for kibana and a webUI query, but some issues happen to
slow-query or out of memory.

Here is my briefly configuration:

shards = 176
replica = 0

my question:

  1. Should I set replica from 0 to 1 ?? As I know, replica means
    distributing data into different nodes, and that will help data recovery.
    But does it helps to increase data searching while ES is doing the query??

  2. Adding more machines is one of the solution, but is there any other
    approaches that I can try to make a better ES configuration?

Thank

Best, Orozco

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/6ac1ac33-d3fd-409f-80a9-fd13abd80ca5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

What version of ES and java are you running?
What is your heap size?
What sort of queries are you running?

Replicas and extra nodes may/will help but a bit more info would clear
things up :slight_smile:

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 18 September 2014 17:54, Hsu Orozco orozcohsu@gmail.com wrote:

Hi all,

I have 4 nodes, memory size respe respectively 64GB, 64GB, 64GB and 128 GB
for handing 8,577,271,728 Documents and 68 TB data

I open that API for kibana and a webUI query, but some issues happen to
slow-query or out of memory.

Here is my briefly configuration:

shards = 176
replica = 0

my question:

  1. Should I set replica from 0 to 1 ?? As I know, replica means
    distributing data into different nodes, and that will help data recovery.
    But does it helps to increase data searching while ES is doing the query??

  2. Adding more machines is one of the solution, but is there any other
    approaches that I can try to make a better ES configuration?

Thank

Best, Orozco

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/6ac1ac33-d3fd-409f-80a9-fd13abd80ca5%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/6ac1ac33-d3fd-409f-80a9-fd13abd80ca5%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624bdcrWenn3DgwsacXaNVQBbapbE0QokpiLAd_%3D19h%2BWkw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

Hi Mark,

I use command GET http://xxx.xxx.xxx.xxx:9200/_nodes to get the one node
information

direct_max_in_bytes=68518871040
heap_init_in_bytes=68719476736
heap_max_in_bytes=68518871040
non_heap_init_in_bytes=24313856
non_heap_max_in_bytes=136314880

total_in_bytes=135366918144

Java version=1.7.0_51
ES version=1.1.1

PS: I just think it's good to separate ES(indexer) and Mysql(data)? I mean
ES only saves the indices and data stores in the Mysql or other powerful
database.
If so, that will increase the query performance (aggregation query)

Thanks a lot

Mark Walkom於 2014年9月18日星期四UTC+8下午3時58分58秒寫道：

What version of ES and java are you running?
What is your heap size?
What sort of queries are you running?

Replicas and extra nodes may/will help but a bit more info would clear
things up :slight_smile:

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com <javascript:>
web: www.campaignmonitor.com

On 18 September 2014 17:54, Hsu Orozco <oroz...@gmail.com <javascript:>>
wrote:

Hi all,

I have 4 nodes, memory size respe respectively 64GB, 64GB, 64GB and 128
GB
for handing 8,577,271,728 Documents and 68 TB data

I open that API for kibana and a webUI query, but some issues happen to
slow-query or out of memory.

Here is my briefly configuration:

shards = 176
replica = 0

my question:

  1. Should I set replica from 0 to 1 ?? As I know, replica means
    distributing data into different nodes, and that will help data recovery.
    But does it helps to increase data searching while ES is doing the query??

  2. Adding more machines is one of the solution, but is there any other
    approaches that I can try to make a better ES configuration?

Thank

Best, Orozco

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/6ac1ac33-d3fd-409f-80a9-fd13abd80ca5%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/6ac1ac33-d3fd-409f-80a9-fd13abd80ca5%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a6840a06-e5a5-4e7c-a43f-01f418d4156a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.