How to choose the number of shards?


(Jonathan Spooner) #1

Are there any documents out there that describe sharding best practices?

I have 36 million records in 1 index with 20 shards, 3 replicated, and 10
nodes. I'm not sure why the previous developer decided on 20 shards and
we've see problems since the index grew from 26 to 36 million.

Thanks,

--


(Shay Banon) #2

I have a whole talk dedicated to it, have a look at it?: http://www.elasticsearch.org/videos/2012/06/05/big-data-search-and-analytics.html

On Oct 20, 2012, at 9:05 PM, Jonathan Spooner jspooner@gmail.com wrote:

Are there any documents out there that describe sharding best practices?

I have 36 million records in 1 index with 20 shards, 3 replicated, and 10 nodes. I'm not sure why the previous developer decided on 20 shards and we've see problems since the index grew from 26 to 36 million.

Thanks,

--

--


(udy brill) #3

Hi,

This link is broken. any chance for a new reference ?

On Sunday, October 21, 2012 12:23:50 AM UTC+2, kimchy wrote:

I have a whole talk dedicated to it, have a look at it?:
http://www.elasticsearch.org/videos/2012/06/05/big-data-search-and-analytics.html

On Oct 20, 2012, at 9:05 PM, Jonathan Spooner <jspo...@gmail.com<javascript:>>
wrote:

Are there any documents out there that describe sharding best practices?

I have 36 million records in 1 index with 20 shards, 3 replicated, and
10 nodes. I'm not sure why the previous developer decided on 20 shards and
we've see problems since the index grew from 26 to 36 million.

Thanks,

--

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1f16f3be-1455-47e8-a467-d2570750b7e3%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Mark Walkom) #4

http://www.elasticsearch.org/videos/big-data-search-and-analytics/

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 17 December 2013 19:05, udybrill@gmail.com wrote:

Hi,

This link is broken. any chance for a new reference ?

On Sunday, October 21, 2012 12:23:50 AM UTC+2, kimchy wrote:

I have a whole talk dedicated to it, have a look at it?:
http://www.elasticsearch.org/videos/2012/06/05/big-data-
search-and-analytics.html

On Oct 20, 2012, at 9:05 PM, Jonathan Spooner jspo...@gmail.com wrote:

Are there any documents out there that describe sharding best
practices?

I have 36 million records in 1 index with 20 shards, 3 replicated, and
10 nodes. I'm not sure why the previous developer decided on 20 shards and
we've see problems since the index grew from 26 to 36 million.

Thanks,

--

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1f16f3be-1455-47e8-a467-d2570750b7e3%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624YFQmM%2BBHOa75T_hHoOfKG6A4GowJ7%2BtxabujA2XYk3jQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #5