Elasticsearch Deployment architecture

Elastic Stack Elasticsearch
1

Hi,

I am very new to elasticsearch, I am trying to deploy elasticsearch in my
dev environment - While there are many ways in which Elasticsearch can be
deployed, I and my team have arrived at this architecture

4 Data Nodes
3 Master Nodes
2 Search Load Balancers (SLB)

Now my question is:

  • Does it make sense to have SLB at all?
  • Can I just have master nodes and have them perform the JOB of SLB too?

Please enlighten me on a sensible Elasticsearch Architecture!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/82ee8ae2-c84d-4685-b061-d3e433b7969f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

1 - Yes if you do a lot of intense querying, eg facets.
2 - Yes that's perfectly fine.

We have a 20 node cluster, with 3 master only nodes that also act as our
search nodes, and the setup works great for us. We may need to revisit this
in the future though.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 29 April 2014 20:40, Dinesh Chandra shadow.on.fire@gmail.com wrote:

Hi,

I am very new to elasticsearch, I am trying to deploy elasticsearch in my
dev environment - While there are many ways in which Elasticsearch can be
deployed, I and my team have arrived at this architecture

4 Data Nodes
3 Master Nodes
2 Search Load Balancers (SLB)

Now my question is:

  • Does it make sense to have SLB at all?
  • Can I just have master nodes and have them perform the JOB of SLB too?

Please enlighten me on a sensible Elasticsearch Architecture!

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/82ee8ae2-c84d-4685-b061-d3e433b7969f%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/82ee8ae2-c84d-4685-b061-d3e433b7969f%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624YC%2B6Yawwxn3j9uy9WwF81pii2isA4%2BECzLyemcfxN%3D4Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

Hi Mark

Thanks for such a quick response !

And thanks also for clarifying the questions ... !

On Tuesday, 29 April 2014 16:12:16 UTC+5:30, Mark Walkom wrote:

1 - Yes if you do a lot of intense querying, eg facets.
2 - Yes that's perfectly fine.

We have a 20 node cluster, with 3 master only nodes that also act as our
search nodes, and the setup works great for us. We may need to revisit this
in the future though.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com <javascript:>
web: www.campaignmonitor.com

On 29 April 2014 20:40, Dinesh Chandra <shadow....@gmail.com <javascript:>

wrote:

Hi,

I am very new to elasticsearch, I am trying to deploy elasticsearch in my
dev environment - While there are many ways in which Elasticsearch can be
deployed, I and my team have arrived at this architecture

4 Data Nodes
3 Master Nodes
2 Search Load Balancers (SLB)

Now my question is:

  • Does it make sense to have SLB at all?
  • Can I just have master nodes and have them perform the JOB of SLB too?

Please enlighten me on a sensible Elasticsearch Architecture!

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/82ee8ae2-c84d-4685-b061-d3e433b7969f%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/82ee8ae2-c84d-4685-b061-d3e433b7969f%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/cc74b51f-62eb-4c33-9fb9-74fb8adf3130%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

4

On a related note, if you have separate slb and master, your main LB (say,
haproxy) would be pointing to the slb , not the master , right?
On 29/04/2014 8:40 pm, "Dinesh Chandra" shadow.on.fire@gmail.com wrote:

Hi,

I am very new to elasticsearch, I am trying to deploy elasticsearch in my
dev environment - While there are many ways in which Elasticsearch can be
deployed, I and my team have arrived at this architecture

4 Data Nodes
3 Master Nodes
2 Search Load Balancers (SLB)

Now my question is:

  • Does it make sense to have SLB at all?
  • Can I just have master nodes and have them perform the JOB of SLB too?

Please enlighten me on a sensible Elasticsearch Architecture!

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/82ee8ae2-c84d-4685-b061-d3e433b7969f%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/82ee8ae2-c84d-4685-b061-d3e433b7969f%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CACj2-4K9mh%3D%3Dv02mkRForLfHO8E4MYUcd3kNvfvFJGWvRwFiCg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

5

For searches, yes. You'd want the indexing to go to the masters.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 30 April 2014 09:02, Norberto Meijome numard@gmail.com wrote:

On a related note, if you have separate slb and master, your main LB (say,
haproxy) would be pointing to the slb , not the master , right?
On 29/04/2014 8:40 pm, "Dinesh Chandra" shadow.on.fire@gmail.com wrote:

Hi,

I am very new to elasticsearch, I am trying to deploy elasticsearch in my
dev environment - While there are many ways in which Elasticsearch can be
deployed, I and my team have arrived at this architecture

4 Data Nodes
3 Master Nodes
2 Search Load Balancers (SLB)

Now my question is:

  • Does it make sense to have SLB at all?
  • Can I just have master nodes and have them perform the JOB of SLB too?

Please enlighten me on a sensible Elasticsearch Architecture!

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/82ee8ae2-c84d-4685-b061-d3e433b7969f%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/82ee8ae2-c84d-4685-b061-d3e433b7969f%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CACj2-4K9mh%3D%3Dv02mkRForLfHO8E4MYUcd3kNvfvFJGWvRwFiCg%40mail.gmail.comhttps://groups.google.com/d/msgid/elasticsearch/CACj2-4K9mh%3D%3Dv02mkRForLfHO8E4MYUcd3kNvfvFJGWvRwFiCg%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624bQXpsN12dCPQefkkL8LMX0bdsGVrs2uS0ZRLMtqRM%3DXg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

6

Sending indexing requests to SLB - is this less optimal, or would outright
fail?
On 30/04/2014 9:04 am, "Mark Walkom" markw@campaignmonitor.com wrote:

For searches, yes. You'd want the indexing to go to the masters.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 30 April 2014 09:02, Norberto Meijome numard@gmail.com wrote:

On a related note, if you have separate slb and master, your main LB
(say, haproxy) would be pointing to the slb , not the master , right?
On 29/04/2014 8:40 pm, "Dinesh Chandra" shadow.on.fire@gmail.com
wrote:

Hi,

I am very new to elasticsearch, I am trying to deploy elasticsearch in
my dev environment - While there are many ways in which Elasticsearch can
be deployed, I and my team have arrived at this architecture

4 Data Nodes
3 Master Nodes
2 Search Load Balancers (SLB)

Now my question is:

  • Does it make sense to have SLB at all?
  • Can I just have master nodes and have them perform the JOB of SLB too?

Please enlighten me on a sensible Elasticsearch Architecture!

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/82ee8ae2-c84d-4685-b061-d3e433b7969f%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/82ee8ae2-c84d-4685-b061-d3e433b7969f%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CACj2-4K9mh%3D%3Dv02mkRForLfHO8E4MYUcd3kNvfvFJGWvRwFiCg%40mail.gmail.comhttps://groups.google.com/d/msgid/elasticsearch/CACj2-4K9mh%3D%3Dv02mkRForLfHO8E4MYUcd3kNvfvFJGWvRwFiCg%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAEM624bQXpsN12dCPQefkkL8LMX0bdsGVrs2uS0ZRLMtqRM%3DXg%40mail.gmail.comhttps://groups.google.com/d/msgid/elasticsearch/CAEM624bQXpsN12dCPQefkkL8LMX0bdsGVrs2uS0ZRLMtqRM%3DXg%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CACj2-4JqQ3Q%3DTKaTWbZTEkbFBW%2Bj6acGeFiBo7omUH-6aEo1Lg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

7

It will work, but if you want to maintain HA then it'd make sense to keep
your inputs separate from your outputs. At least, that's my take :slight_smile:

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 30 April 2014 19:48, Norberto Meijome numard@gmail.com wrote:

Sending indexing requests to SLB - is this less optimal, or would outright
fail?
On 30/04/2014 9:04 am, "Mark Walkom" markw@campaignmonitor.com wrote:

For searches, yes. You'd want the indexing to go to the masters.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 30 April 2014 09:02, Norberto Meijome numard@gmail.com wrote:

On a related note, if you have separate slb and master, your main LB
(say, haproxy) would be pointing to the slb , not the master , right?
On 29/04/2014 8:40 pm, "Dinesh Chandra" shadow.on.fire@gmail.com
wrote:

Hi,

I am very new to elasticsearch, I am trying to deploy elasticsearch in
my dev environment - While there are many ways in which Elasticsearch can
be deployed, I and my team have arrived at this architecture

4 Data Nodes
3 Master Nodes
2 Search Load Balancers (SLB)

Now my question is:

  • Does it make sense to have SLB at all?
  • Can I just have master nodes and have them perform the JOB of SLB
    too?

Please enlighten me on a sensible Elasticsearch Architecture!

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/82ee8ae2-c84d-4685-b061-d3e433b7969f%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/82ee8ae2-c84d-4685-b061-d3e433b7969f%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CACj2-4K9mh%3D%3Dv02mkRForLfHO8E4MYUcd3kNvfvFJGWvRwFiCg%40mail.gmail.comhttps://groups.google.com/d/msgid/elasticsearch/CACj2-4K9mh%3D%3Dv02mkRForLfHO8E4MYUcd3kNvfvFJGWvRwFiCg%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAEM624bQXpsN12dCPQefkkL8LMX0bdsGVrs2uS0ZRLMtqRM%3DXg%40mail.gmail.comhttps://groups.google.com/d/msgid/elasticsearch/CAEM624bQXpsN12dCPQefkkL8LMX0bdsGVrs2uS0ZRLMtqRM%3DXg%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CACj2-4JqQ3Q%3DTKaTWbZTEkbFBW%2Bj6acGeFiBo7omUH-6aEo1Lg%40mail.gmail.comhttps://groups.google.com/d/msgid/elasticsearch/CACj2-4JqQ3Q%3DTKaTWbZTEkbFBW%2Bj6acGeFiBo7omUH-6aEo1Lg%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624Y9ZS7FAVa1R%3D-sK4UWyJj6uoSC756fe%3Dii7Xi2e7Kn0A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.