# Designing an index that holds updating product data feeds

**URL:** https://discuss.elastic.co/t/designing-an-index-that-holds-updating-product-data-feeds/20547
**Category:** Elasticsearch
**Created:** [November 3, 2014, 9:31pm UTC](https://discuss.elastic.co/t/designing-an-index-that-holds-updating-product-data-feeds/20547 "2014-11-03T21:31:30Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![Ori\_P](https://avatars.discourse-cdn.com/v4/letter/o/3bc359/32.png) [@Ori\_P](https://discuss.elastic.co/u/Ori_P)
#### Post date: [November 3, 2014, 9:31pm UTC](https://discuss.elastic.co/t/designing-an-index-that-holds-updating-product-data-feeds/20547/1 "2014-11-03T21:31:30Z")

</div>

I would appreciate your suggestions in helping me design my elasticsearch  
index.

I'm intending to index product feeds from about 20 on-line stores, each  
store not having more than 20,000 products. each product has about 15 basic  
fields.  
Most of the searches would be done on specific product categories, and not  
specific stores.

Each store feed is updated every few days (each store separately), by  
receiving an XML file containing all the products in the store (no deltas).  
Each update, I need to remove from my index all the existing products from  
that store and add the new ones.

I thought of two possibles approaches:

1. Create a single index + an alias to that index. Once a new feed is  
received, clone the existing index to a new index, remove from the new  
index all the old products, add the new products and finally change the  
alias to point to the new index.

2. Create an index for each store, and an alias that points to all of the  
indices. Once a new feed is received, just index it from scratch, remove  
the old store index from the alias and add the new one.

I'm not sure which way will give me faster search results? or maybe there  
is an even better approach I didn't think of...

Thanks in advance,

Ori

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/34f2766d-cada-4ba9-a4fa-961c34aa2f8b%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/34f2766d-cada-4ba9-a4fa-961c34aa2f8b%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [November 3, 2014, 9:54pm UTC](https://discuss.elastic.co/t/designing-an-index-that-holds-updating-product-data-feeds/20547/2 "2014-11-03T21:54:15Z")

</div>

I don't see any benefit of solution 1.

I would definitely do solution 2.

I don't really think you could see a difference search time wise. But in term of IO 2 is better.  
Also, you should modify refresh interval while indexing to -1 and call refresh after the bulk load.

HTH

--  
David 😉  
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

> Le 3 nov. 2014 à 21:31, Ori P [shalti@gmail.com](mailto:shalti@gmail.com) a écrit :
> 
> I would appreciate your suggestions in helping me design my elasticsearch index.
> 
> I'm intending to index product feeds from about 20 on-line stores, each store not having more than 20,000 products. each product has about 15 basic fields.  
> Most of the searches would be done on specific product categories, and not specific stores.
> 
> Each store feed is updated every few days (each store separately), by receiving an XML file containing all the products in the store (no deltas). Each update, I need to remove from my index all the existing products from that store and add the new ones.
> 
> I thought of two possibles approaches:
> 
> 1. Create a single index + an alias to that index. Once a new feed is received, clone the existing index to a new index, remove from the new index all the old products, add the new products and finally change the alias to point to the new index.
> 
> 2. Create an index for each store, and an alias that points to all of the indices. Once a new feed is received, just index it from scratch, remove the old store index from the alias and add the new one.
> 
> I'm not sure which way will give me faster search results? or maybe there is an even better approach I didn't think of...
> 
> Thanks in advance,
> 
> ## Ori
> 
> You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/34f2766d-cada-4ba9-a4fa-961c34aa2f8b%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/34f2766d-cada-4ba9-a4fa-961c34aa2f8b%40googlegroups.com).  
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/AF883E98-1AD1-4309-8062-19CFF9EAA246%40pilato.fr](https://groups.google.com/d/msgid/elasticsearch/AF883E98-1AD1-4309-8062-19CFF9EAA246%40pilato.fr).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

### Author: ![Ori\_P](https://avatars.discourse-cdn.com/v4/letter/o/3bc359/32.png) [@Ori\_P](https://discuss.elastic.co/u/Ori_P)
#### Post date: [November 3, 2014, 10:01pm UTC](https://discuss.elastic.co/t/designing-an-index-that-holds-updating-product-data-feeds/20547/3 "2014-11-03T22:01:30Z")

</div>

Thanks for replying David.

I thought approach 2 might be problematic since the alias on multiple  
indices would cause a query to run on every index separately, which I  
thought might slow things down. Apparently I was wrong?

And thanks for the tip about the refresh interval 🙂

On Monday, November 3, 2014 11:54:38 PM UTC+2, David Pilato wrote:

> I don't see any benefit of solution 1.
> 
> I would definitely do solution 2.
> 
> I don't really think you could see a difference search time wise. But in  
> term of IO 2 is better.  
> Also, you should modify refresh interval while indexing to -1 and call  
> refresh after the bulk load.
> 
> HTH
> 
> --  
> David 😉  
> Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
> 
> Le 3 nov. 2014 à 21:31, Ori P \<[sha...@gmail.com](mailto:sha...@gmail.com) \<javascript:\>\> a écrit :
> 
> I would appreciate your suggestions in helping me design my elasticsearch  
> index.
> 
> I'm intending to index product feeds from about 20 on-line stores, each  
> store not having more than 20,000 products. each product has about 15 basic  
> fields.  
> Most of the searches would be done on specific product categories, and not  
> specific stores.
> 
> Each store feed is updated every few days (each store separately), by  
> receiving an XML file containing all the products in the store (no deltas).  
> Each update, I need to remove from my index all the existing products from  
> that store and add the new ones.
> 
> I thought of two possibles approaches:
> 
> 1. Create a single index + an alias to that index. Once a new feed is  
> received, clone the existing index to a new index, remove from the new  
> index all the old products, add the new products and finally change the  
> alias to point to the new index.
> 
> 2. Create an index for each store, and an alias that points to all of the  
> indices. Once a new feed is received, just index it from scratch, remove  
> the old store index from the alias and add the new one.
> 
> I'm not sure which way will give me faster search results? or maybe there  
> is an even better approach I didn't think of...
> 
> Thanks in advance,
> 
> Ori
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \<javascript:\>.  
> To view this discussion on the web visit  
> [https://groups.google.com/d/msgid/elasticsearch/34f2766d-cada-4ba9-a4fa-961c34aa2f8b%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/34f2766d-cada-4ba9-a4fa-961c34aa2f8b%40googlegroups.com)  
> [https://groups.google.com/d/msgid/elasticsearch/34f2766d-cada-4ba9-a4fa-961c34aa2f8b%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/34f2766d-cada-4ba9-a4fa-961c34aa2f8b%40googlegroups.com?utm_medium=email&utm_source=footer)  
> .  
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/6c85ec37-e93e-47d6-a29f-72207f9925d8%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6c85ec37-e93e-47d6-a29f-72207f9925d8%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [November 3, 2014, 10:21pm UTC](https://discuss.elastic.co/t/designing-an-index-that-holds-updating-product-data-feeds/20547/4 "2014-11-03T22:21:28Z")

</div>

Hmmm. Sounds like I misread what you explained in 2.

I missed the fact you want to have one index per store. So let me change my answer.  
If a single index, one shard, can hold your 400 000 docs which sounds reasonable to me, then one single index will be faster than querying 20 indices.

My 2 cents

--  
David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com)  
@dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr [https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr) | @scrutmydocs [https://twitter.com/scrutmydocs](https://twitter.com/scrutmydocs)

> Le 3 nov. 2014 à 23:01, Ori P [shalti@gmail.com](mailto:shalti@gmail.com) a écrit :
> 
> Thanks for replying David.
> 
> I thought approach 2 might be problematic since the alias on multiple indices would cause a query to run on every index separately, which I thought might slow things down. Apparently I was wrong?
> 
> And thanks for the tip about the refresh interval 🙂
> 
> On Monday, November 3, 2014 11:54:38 PM UTC+2, David Pilato wrote:  
> I don't see any benefit of solution 1.
> 
> I would definitely do solution 2.
> 
> I don't really think you could see a difference search time wise. But in term of IO 2 is better.  
> Also, you should modify refresh interval while indexing to -1 and call refresh after the bulk load.
> 
> HTH
> 
> --  
> David 😉  
> Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
> 
> Le 3 nov. 2014 à 21:31, Ori P \<[sha...@gmail.com](mailto:sha...@gmail.com) \<javascript:\>\> a écrit :
> 
> > I would appreciate your suggestions in helping me design my elasticsearch index.
> > 
> > I'm intending to index product feeds from about 20 on-line stores, each store not having more than 20,000 products. each product has about 15 basic fields.  
> > Most of the searches would be done on specific product categories, and not specific stores.
> > 
> > Each store feed is updated every few days (each store separately), by receiving an XML file containing all the products in the store (no deltas). Each update, I need to remove from my index all the existing products from that store and add the new ones.
> > 
> > I thought of two possibles approaches:
> > 
> > 1. Create a single index + an alias to that index. Once a new feed is received, clone the existing index to a new index, remove from the new index all the old products, add the new products and finally change the alias to point to the new index.
> > 
> > 2. Create an index for each store, and an alias that points to all of the indices. Once a new feed is received, just index it from scratch, remove the old store index from the alias and add the new one.
> > 
> > I'm not sure which way will give me faster search results? or maybe there is an even better approach I didn't think of...
> > 
> > Thanks in advance,
> > 
> > Ori
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \<javascript:\>.  
> > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/34f2766d-cada-4ba9-a4fa-961c34aa2f8b%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/34f2766d-cada-4ba9-a4fa-961c34aa2f8b%40googlegroups.com) [https://groups.google.com/d/msgid/elasticsearch/34f2766d-cada-4ba9-a4fa-961c34aa2f8b%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/34f2766d-cada-4ba9-a4fa-961c34aa2f8b%40googlegroups.com?utm_medium=email&utm_source=footer).  
> > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout) [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> 
> --  
> You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) [mailto:elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/6c85ec37-e93e-47d6-a29f-72207f9925d8%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6c85ec37-e93e-47d6-a29f-72207f9925d8%40googlegroups.com) [https://groups.google.com/d/msgid/elasticsearch/6c85ec37-e93e-47d6-a29f-72207f9925d8%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/6c85ec37-e93e-47d6-a29f-72207f9925d8%40googlegroups.com?utm_medium=email&utm_source=footer).  
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout) [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/4CB2DC5E-6512-4933-BA26-DDE45792D531%40pilato.fr](https://groups.google.com/d/msgid/elasticsearch/4CB2DC5E-6512-4933-BA26-DDE45792D531%40pilato.fr).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

### Author: ![Ori\_P](https://avatars.discourse-cdn.com/v4/letter/o/3bc359/32.png) [@Ori\_P](https://discuss.elastic.co/u/Ori_P)
#### Post date: [November 3, 2014, 10:43pm UTC](https://discuss.elastic.co/t/designing-an-index-that-holds-updating-product-data-feeds/20547/5 "2014-11-03T22:43:58Z")

</div>

And if I may ask, do you have a suggestion on how to update the single  
index? I need to replace on a daily basis a bulk of about 20,000 documents  
at once, with as little performance and data availability implications as  
possible.

On Tuesday, November 4, 2014 12:21:51 AM UTC+2, David Pilato wrote:

> Hmmm. Sounds like I misread what you explained in 2.
> 
> I missed the fact you want to have one index per store. So let me change  
> my answer.  
> If a single index, one shard, can hold your 400 000 docs which sounds  
> reasonable to me, then one single index will be faster than querying 20  
> indices.
> 
> My 2 cents
> 
> --  
> _David Pilato_ | _Technical Advocate_ | _[Elasticsearch.com](http://Elasticsearch.com)  
> [http://Elasticsearch.com](http://Elasticsearch.com)_  
> @dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr  
> [https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr) | @scrutmydocs  
> [https://twitter.com/scrutmydocs](https://twitter.com/scrutmydocs)
> 
> Le 3 nov. 2014 à 23:01, Ori P \<[sha...@gmail.com](mailto:sha...@gmail.com) \<javascript:\>\> a écrit :
> 
> Thanks for replying David.
> 
> I thought approach 2 might be problematic since the alias on multiple  
> indices would cause a query to run on every index separately, which I  
> thought might slow things down. Apparently I was wrong?
> 
> And thanks for the tip about the refresh interval 🙂
> 
> On Monday, November 3, 2014 11:54:38 PM UTC+2, David Pilato wrote:
> 
> > I don't see any benefit of solution 1.
> > 
> > I would definitely do solution 2.
> > 
> > I don't really think you could see a difference search time wise. But in  
> > term of IO 2 is better.  
> > Also, you should modify refresh interval while indexing to -1 and call  
> > refresh after the bulk load.
> > 
> > HTH
> > 
> > --  
> > David 😉  
> > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
> > 
> > Le 3 nov. 2014 à 21:31, Ori P [sha...@gmail.com](mailto:sha...@gmail.com) a écrit :
> > 
> > I would appreciate your suggestions in helping me design my elasticsearch  
> > index.
> > 
> > I'm intending to index product feeds from about 20 on-line stores, each  
> > store not having more than 20,000 products. each product has about 15 basic  
> > fields.  
> > Most of the searches would be done on specific product categories, and  
> > not specific stores.
> > 
> > Each store feed is updated every few days (each store separately), by  
> > receiving an XML file containing all the products in the store (no deltas).  
> > Each update, I need to remove from my index all the existing products from  
> > that store and add the new ones.
> > 
> > I thought of two possibles approaches:
> > 
> > 1. Create a single index + an alias to that index. Once a new feed is  
> > received, clone the existing index to a new index, remove from the new  
> > index all the old products, add the new products and finally change the  
> > alias to point to the new index.
> > 
> > 2. Create an index for each store, and an alias that points to all of the  
> > indices. Once a new feed is received, just index it from scratch, remove  
> > the old store index from the alias and add the new one.
> > 
> > I'm not sure which way will give me faster search results? or maybe there  
> > is an even better approach I didn't think of...
> > 
> > Thanks in advance,
> > 
> > Ori
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups  
> > "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an  
> > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).  
> > To view this discussion on the web visit  
> > [https://groups.google.com/d/msgid/elasticsearch/34f2766d-cada-4ba9-a4fa-961c34aa2f8b%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/34f2766d-cada-4ba9-a4fa-961c34aa2f8b%40googlegroups.com)  
> > [https://groups.google.com/d/msgid/elasticsearch/34f2766d-cada-4ba9-a4fa-961c34aa2f8b%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/34f2766d-cada-4ba9-a4fa-961c34aa2f8b%40googlegroups.com?utm_medium=email&utm_source=footer)  
> > .  
> > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \<javascript:\>.  
> To view this discussion on the web visit  
> [https://groups.google.com/d/msgid/elasticsearch/6c85ec37-e93e-47d6-a29f-72207f9925d8%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6c85ec37-e93e-47d6-a29f-72207f9925d8%40googlegroups.com)  
> [https://groups.google.com/d/msgid/elasticsearch/6c85ec37-e93e-47d6-a29f-72207f9925d8%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/6c85ec37-e93e-47d6-a29f-72207f9925d8%40googlegroups.com?utm_medium=email&utm_source=footer)  
> .  
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/6e4d869d-f09b-4f20-b2ca-4639c4a7bab4%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6e4d869d-f09b-4f20-b2ca-4639c4a7bab4%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [November 3, 2014, 11:43pm UTC](https://discuss.elastic.co/t/designing-an-index-that-holds-updating-product-data-feeds/20547/6 "2014-11-03T23:43:21Z")

</div>

Well. I’m use to run demo where I can inject on my laptop (SSD drives) around 8k to 10k doc per second.  
I think the biggest problem you can have is to read your source documents not to write them to elasticsearch.

With a single index, I would probably reindex the 400 000 docs every day in a new a clean index and then switch the alias from old to new index.

But it depends on your read rate I guess.

--  
David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com)  
@dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr [https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr) | @scrutmydocs [https://twitter.com/scrutmydocs](https://twitter.com/scrutmydocs)

> Le 3 nov. 2014 à 23:43, Ori P [shalti@gmail.com](mailto:shalti@gmail.com) a écrit :
> 
> And if I may ask, do you have a suggestion on how to update the single index? I need to replace on a daily basis a bulk of about 20,000 documents at once, with as little performance and data availability implications as possible.
> 
> On Tuesday, November 4, 2014 12:21:51 AM UTC+2, David Pilato wrote:  
> Hmmm. Sounds like I misread what you explained in 2.
> 
> I missed the fact you want to have one index per store. So let me change my answer.  
> If a single index, one shard, can hold your 400 000 docs which sounds reasonable to me, then one single index will be faster than querying 20 indices.
> 
> My 2 cents
> 
> --  
> David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com) [http://elasticsearch.com/](http://elasticsearch.com/)  
> @dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr [https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr) | @scrutmydocs [https://twitter.com/scrutmydocs](https://twitter.com/scrutmydocs)
> 
> > Le 3 nov. 2014 à 23:01, Ori P \<[sha...@gmail.com](mailto:sha...@gmail.com) \<javascript:\>\> a écrit :
> > 
> > Thanks for replying David.
> > 
> > I thought approach 2 might be problematic since the alias on multiple indices would cause a query to run on every index separately, which I thought might slow things down. Apparently I was wrong?
> > 
> > And thanks for the tip about the refresh interval 🙂
> > 
> > On Monday, November 3, 2014 11:54:38 PM UTC+2, David Pilato wrote:  
> > I don't see any benefit of solution 1.
> > 
> > I would definitely do solution 2.
> > 
> > I don't really think you could see a difference search time wise. But in term of IO 2 is better.  
> > Also, you should modify refresh interval while indexing to -1 and call refresh after the bulk load.
> > 
> > HTH
> > 
> > --  
> > David 😉  
> > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
> > 
> > Le 3 nov. 2014 à 21:31, Ori P \<[sha...@gmail.com](mailto:sha...@gmail.com) \<\>\> a écrit :
> > 
> > > I would appreciate your suggestions in helping me design my elasticsearch index.
> > > 
> > > I'm intending to index product feeds from about 20 on-line stores, each store not having more than 20,000 products. each product has about 15 basic fields.  
> > > Most of the searches would be done on specific product categories, and not specific stores.
> > > 
> > > Each store feed is updated every few days (each store separately), by receiving an XML file containing all the products in the store (no deltas). Each update, I need to remove from my index all the existing products from that store and add the new ones.
> > > 
> > > I thought of two possibles approaches:
> > > 
> > > 1. Create a single index + an alias to that index. Once a new feed is received, clone the existing index to a new index, remove from the new index all the old products, add the new products and finally change the alias to point to the new index.
> > > 
> > > 2. Create an index for each store, and an alias that points to all of the indices. Once a new feed is received, just index it from scratch, remove the old store index from the alias and add the new one.
> > > 
> > > I'm not sure which way will give me faster search results? or maybe there is an even better approach I didn't think of...
> > > 
> > > Thanks in advance,
> > > 
> > > Ori
> > > 
> > > --  
> > > You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
> > > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \<\>.  
> > > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/34f2766d-cada-4ba9-a4fa-961c34aa2f8b%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/34f2766d-cada-4ba9-a4fa-961c34aa2f8b%40googlegroups.com) [https://groups.google.com/d/msgid/elasticsearch/34f2766d-cada-4ba9-a4fa-961c34aa2f8b%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/34f2766d-cada-4ba9-a4fa-961c34aa2f8b%40googlegroups.com?utm_medium=email&utm_source=footer).  
> > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout) [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \<javascript:\>.  
> > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/6c85ec37-e93e-47d6-a29f-72207f9925d8%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6c85ec37-e93e-47d6-a29f-72207f9925d8%40googlegroups.com) [https://groups.google.com/d/msgid/elasticsearch/6c85ec37-e93e-47d6-a29f-72207f9925d8%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/6c85ec37-e93e-47d6-a29f-72207f9925d8%40googlegroups.com?utm_medium=email&utm_source=footer).  
> > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout) [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> 
> --  
> You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) [mailto:elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/6e4d869d-f09b-4f20-b2ca-4639c4a7bab4%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6e4d869d-f09b-4f20-b2ca-4639c4a7bab4%40googlegroups.com) [https://groups.google.com/d/msgid/elasticsearch/6e4d869d-f09b-4f20-b2ca-4639c4a7bab4%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/6e4d869d-f09b-4f20-b2ca-4639c4a7bab4%40googlegroups.com?utm_medium=email&utm_source=footer).  
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout) [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/3AABA50C-DAED-4BB9-B14A-C178C1D0CBE5%40pilato.fr](https://groups.google.com/d/msgid/elasticsearch/3AABA50C-DAED-4BB9-B14A-C178C1D0CBE5%40pilato.fr).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 12:52am UTC](https://discuss.elastic.co/t/designing-an-index-that-holds-updating-product-data-feeds/20547/7 "2017-07-06T00:52:24Z")

</div>


