ES seems slow and ineffecient


(Dan Storm) #1

I have a cluster setup with three nodes.
The three nodes are setup within LXC with 8GB of ram dedicated to each
container. Heap size is set to 4GB on each and bootstrap.mlockall is set to
true.

iptables has been setup to forward incoming traffic on port 9200 to the
master node.

Heres an image of the diagnostics provided by HQ.

http://i.imgur.com/z59W4hV.png
I experience quite a few timeouts.
The dedicated server is a 48GB RAM setup with 8 cores and SSD disks, so I
would believe I have enough power to handle the load.

Can anyone see if there is something I should be aware of or something that
I can do to make my

Thank you.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/537e0142-a726-44f8-a1fb-e11faa51c052%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Mark Walkom) #2

Can you elaborate on the timeouts?

What Elasticsearch version, what java version?

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 30 June 2014 18:40, Dan Storm l33tnasty@gmail.com wrote:

I have a cluster setup with three nodes.
The three nodes are setup within LXC with 8GB of ram dedicated to each
container. Heap size is set to 4GB on each and bootstrap.mlockall is set to
true.

iptables has been setup to forward incoming traffic on port 9200 to the
master node.

Heres an image of the diagnostics provided by HQ.

http://i.imgur.com/z59W4hV.png
I experience quite a few timeouts.
The dedicated server is a 48GB RAM setup with 8 cores and SSD disks, so I
would believe I have enough power to handle the load.

Can anyone see if there is something I should be aware of or something
that I can do to make my

Thank you.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/537e0142-a726-44f8-a1fb-e11faa51c052%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/537e0142-a726-44f8-a1fb-e11faa51c052%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624bYb-EQJyeqJeSZmH-_Jqc5szfSo1mgXuM6wR6hhCJuUg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Dan Storm) #3

Hi Mark,

Java:
java version "1.7.0_55"
OpenJDK Runtime Environment (IcedTea 2.4.7) (7u55-2.4.7-1ubuntu1~0.13.10.1)

ES is version 1.2.1

The timeouts happen when higher number of queries occur and ES has memory
issues. But I'm unsure as to why the setup lacks memory.

On Monday, June 30, 2014 12:26:23 PM UTC+2, Mark Walkom wrote:

Can you elaborate on the timeouts?

What Elasticsearch version, what java version?

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com <javascript:>
web: www.campaignmonitor.com

On 30 June 2014 18:40, Dan Storm <l33t...@gmail.com <javascript:>> wrote:

I have a cluster setup with three nodes.
The three nodes are setup within LXC with 8GB of ram dedicated to each
container. Heap size is set to 4GB on each and bootstrap.mlockall is set to
true.

iptables has been setup to forward incoming traffic on port 9200 to the
master node.

Heres an image of the diagnostics provided by HQ.

http://i.imgur.com/z59W4hV.png
I experience quite a few timeouts.
The dedicated server is a 48GB RAM setup with 8 cores and SSD disks, so I
would believe I have enough power to handle the load.

Can anyone see if there is something I should be aware of or something
that I can do to make my

Thank you.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/537e0142-a726-44f8-a1fb-e11faa51c052%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/537e0142-a726-44f8-a1fb-e11faa51c052%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5faadcf2-7845-4aab-a651-86bb09d5572e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Jörg Prante) #4

If you have one server with 48G RAM / 8 cores / SSD you should se up one
Elasticsearch node because JVM is capable of managing it.

By setting up 3 nodes on just a single server, you only add overhead and
the risk of timeouts and instability is higher.

Jörg

On Mon, Jun 30, 2014 at 10:40 AM, Dan Storm l33tnasty@gmail.com wrote:

I have a cluster setup with three nodes.
The three nodes are setup within LXC with 8GB of ram dedicated to each
container. Heap size is set to 4GB on each and bootstrap.mlockall is set to
true.

iptables has been setup to forward incoming traffic on port 9200 to the
master node.

Heres an image of the diagnostics provided by HQ.

http://i.imgur.com/z59W4hV.png
I experience quite a few timeouts.
The dedicated server is a 48GB RAM setup with 8 cores and SSD disks, so I
would believe I have enough power to handle the load.

Can anyone see if there is something I should be aware of or something
that I can do to make my

Thank you.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/537e0142-a726-44f8-a1fb-e11faa51c052%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/537e0142-a726-44f8-a1fb-e11faa51c052%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHS2tHm2SkFZsVC77PGKASDxfTjfuzkJ_21K-NW0WeEvg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(system) #5