# Using elasticsearch percolate + N indices as a feed system
**URL:** https://discuss.elastic.co/t/using-elasticsearch-percolate-n-indices-as-a-feed-system/18354
**Category:** Elasticsearch
**Created:** [June 26, 2014, 4:59pm UTC](https://discuss.elastic.co/t/using-elasticsearch-percolate-n-indices-as-a-feed-system/18354 "2014-06-26T16:59:46Z")
**Posts on this page:** 2
**Page:** 1
### Author:  [@Yan\_Pritzker](https://discuss.elastic.co/u/Yan_Pritzker)
#### Post date: [June 26, 2014, 4:59pm UTC](https://discuss.elastic.co/t/using-elasticsearch-percolate-n-indices-as-a-feed-system/18354/1 "2014-06-26T16:59:46Z")
Hey all,
We are considering building a fan-out feed inbox type system on top of ES
for [Reverb.com](http://Reverb.com). The way it would work is each user can follow some number
of searches. Using the percolator, we would plop new items as they matched
searches into individual user feeds. We are going to have the problem of
many users following a common search such as "all electric guitars" so
fanouts are going to get potentially expensive.
We're thinking of structuring it so that each user has an index of items
that just contains the item id's and then query against that index and then
again against the main item index to get the details.
I would love some feedback on some of these ideas in terms of feasibility
in ES. We like the idea of building on top of ES to avoid introducing
additional infrastructure or other databases as ES is our primary search
engine.
A couple questions
1. Is this a sane design? Any obvious flaws?
2. We will need to prune items that sell out of everyone's indices. Is the
delete by query API expensive if we have to run a query such as "delete
product from these 1M indices", having an index per user.
3. Does percolation get expensive if we have a million users subscribed to
500k different searches.
4. We could potentially shard the cluster by user id so that users 1-100k
lived on one cluster, etc.
Your help is greatly appreciated
thanks,
Yan Pritzker
> **[Buy & Sell Music Gear Online](https://Reverb.com/)**
>
> Reverb.com is the marketplace for musicians to buy and sell used, vintage, and new music gear online. Welcome to the world’s largest music gear website.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/33feaea6-d59b-4814-9c69-6c9f85408076%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/33feaea6-d59b-4814-9c69-6c9f85408076%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 1:19am UTC](https://discuss.elastic.co/t/using-elasticsearch-percolate-n-indices-as-a-feed-system/18354/2 "2017-07-06T01:19:28Z")