Hi,
Do you have any plans or thoughts on allowing to change
(increase/decrease) the number of shards and replicas of an existing
index? It seems like currently these can only be specified at index
creation time.
--
Bruno Dumon
Outerthought
Yes, certainly, though the effort involved in supporting it is quite big
(its quite complex when putting search engine into the picture, compared
with the very simple models like dynamo, for example).
But, most cases you won't necessarily need it because of the "multi-index"
support elasticsearch provides. For example, lets take a twitter example. We
can decide to create an index per user, or create an index per day. The
ability to perform all operations in elasticsearch across more than one
index is intentional to support that.
This gives you added benefit. For example, if you have an index per user in
the twitter example, you can search on your friends and their friends (and
score differently based on the relationship you have with them) quite
easily.
-shay.banon
On Wed, Feb 24, 2010 at 5:38 PM, Bruno Dumon bruno@outerthought.org wrote:
Hi,
Do you have any plans or thoughts on allowing to change
(increase/decrease) the number of shards and replicas of an existing
index? It seems like currently these can only be specified at index
creation time.--
Bruno Dumon
Outerthought
http://outerthought.org/
One more thing I forgot to mention. Even with one index, and the out of the
box default of 5 shards with 1 replica each, you can scale easily up to 10
machines (just for the index, of course you can create more indices and
scale even more) which gives you hugh amount of storage (10s of terra) and
great search scalability. Also, changing the number of replicas dynamically
should be much simpler. If search scalability is what you are after.
-shay.banon
On Wed, Feb 24, 2010 at 8:27 PM, Shay Banon shay.banon@elasticsearch.comwrote:
Yes, certainly, though the effort involved in supporting it is quite big
(its quite complex when putting search engine into the picture, compared
with the very simple models like dynamo, for example).But, most cases you won't necessarily need it because of the "multi-index"
support elasticsearch provides. For example, lets take a twitter example. We
can decide to create an index per user, or create an index per day. The
ability to perform all operations in elasticsearch across more than one
index is intentional to support that.This gives you added benefit. For example, if you have an index per user in
the twitter example, you can search on your friends and their friends (and
score differently based on the relationship you have with them) quite
easily.-shay.banon
On Wed, Feb 24, 2010 at 5:38 PM, Bruno Dumon bruno@outerthought.orgwrote:
Hi,
Do you have any plans or thoughts on allowing to change
(increase/decrease) the number of shards and replicas of an existing
index? It seems like currently these can only be specified at index
creation time.--
Bruno Dumon
Outerthought
http://outerthought.org/
Thanks for the clarification.
Using multiple indices a a way to scale is an interesting thought, I
had not considered it.
Changing the number of shards probably needs less flexibility than
changing the number of replicas, though it would be nice if there was
some way to grow, even if it would be just by doubling the number of
current shards (i.e. splitting each shard into two).
But anyway, the current flexibility and power of Elasticsearch is
already quite impressive!
On Wed, Feb 24, 2010 at 7:37 PM, Shay Banon
shay.banon@elasticsearch.com wrote:
One more thing I forgot to mention. Even with one index, and the out of the
box default of 5 shards with 1 replica each, you can scale easily up to 10
machines (just for the index, of course you can create more indices and
scale even more) which gives you hugh amount of storage (10s of terra) and
great search scalability. Also, changing the number of replicas dynamically
should be much simpler. If search scalability is what you are after.
-shay.banonOn Wed, Feb 24, 2010 at 8:27 PM, Shay Banon shay.banon@elasticsearch.com
wrote:Yes, certainly, though the effort involved in supporting it is quite big
(its quite complex when putting search engine into the picture, compared
with the very simple models like dynamo, for example).
But, most cases you won't necessarily need it because of the "multi-index"
support elasticsearch provides. For example, lets take a twitter example. We
can decide to create an index per user, or create an index per day. The
ability to perform all operations in elasticsearch across more than one
index is intentional to support that.
This gives you added benefit. For example, if you have an index per user
in the twitter example, you can search on your friends and their friends
(and score differently based on the relationship you have with them) quite
easily.
-shay.banonOn Wed, Feb 24, 2010 at 5:38 PM, Bruno Dumon bruno@outerthought.org
wrote:Hi,
Do you have any plans or thoughts on allowing to change
(increase/decrease) the number of shards and replicas of an existing
index? It seems like currently these can only be specified at index
creation time.
--
Bruno Dumon
Outerthought
http://outerthought.org/