# Query and allocate data to shards based on tags **URL:** \ **Category:** Elasticsearch\ **Created:** [July 3, 2013, 6:01am UTC](https://discuss.elastic.co/t/query-and-allocate-data-to-shards-based-on-tags/12632 "2013-07-03T06:01:14Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![aryan](https://avatars.discourse-cdn.com/v4/letter/a/d78d45/32.png) [@aryan](https://discuss.elastic.co/u/aryan)\ **Post date:** [July 3, 2013, 6:01am UTC](https://discuss.elastic.co/t/query-and-allocate-data-to-shards-based-on-tags/12632/1 "2013-07-03T06:01:14Z")
Hi, I'm fairly new to elasticsearch, so sorry if this is a trivial question. I'm running a typical logstash-redis-elasticsearch system to capture all my logs(around 500 GB/day). To my knowledge elasticsearch queries every shard in an index and aggregates the results, but due to the volume of logs per day and the response times needed, I want to query only few shards which of course should be decided on some "tag" in the message. So I'm looking at a way to allocate data to shards based on some tags and query only relevant shards based on the tags. Any leads, references or solutions on how to achieve this ? I've already looked at shard allocation filtering but that doesn't cater this specific requirement. Thanks, Aryan ---
**Author:** ![Randall\_McRee](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/randall_mcree/32/47177_2.png) [@Randall\_McRee](https://discuss.elastic.co/u/Randall_McRee)\ **Post date:** [July 9, 2013, 11:10pm UTC](https://discuss.elastic.co/t/query-and-allocate-data-to-shards-based-on-tags/12632/2 "2013-07-09T23:10:23Z")
What you want is called "routing" in elasticsearch. Search for it by that name and you will find plenty of information. On Tue, Jul 2, 2013 at 11:01 PM, aryan [abhilashm24@yahoo.com](mailto:abhilashm24@yahoo.com) wrote: > Hi, > > I'm fairly new to elasticsearch, so sorry if this is a trivial question. > > I'm running a typical logstash-redis-elasticsearch system to capture all my > logs(around 500 GB/day). To my knowledge elasticsearch queries every shard > in an index and aggregates the results, but due to the volume of logs per > day and the response times needed, I want to query only few shards which of > course should be decided on some "tag" in the message. So I'm looking at a > way to allocate data to shards based on some tags and query only relevant > shards based on the tags. Any leads, references or solutions on how to > achieve this ? > > I've already looked at shard allocation filtering but that doesn't cater > this specific requirement. > > Thanks, > Aryan > > -- > View this message in context: > [http://elasticsearch-users.115913.n3.nabble.com/Query-and-allocate-data-to-shards-based-on-tags-tp4037420.html](http://elasticsearch-users.115913.n3.nabble.com/Query-and-allocate-data-to-shards-based-on-tags-tp4037420.html) > Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [July 10, 2013, 8:51am UTC](https://discuss.elastic.co/t/query-and-allocate-data-to-shards-based-on-tags/12632/3 "2013-07-10T08:51:39Z")
Hey, another option are time based indices (especially in a log file based setup), as you are searching for a certain time period most of the time. --Alex On Wed, Jul 10, 2013 at 1:10 AM, Randall McRee [randall.mcree@gmail.com](mailto:randall.mcree@gmail.com)wrote: > What you want is called "routing" in elasticsearch. Search for it by that > name and you will find plenty of information. > > On Tue, Jul 2, 2013 at 11:01 PM, aryan [abhilashm24@yahoo.com](mailto:abhilashm24@yahoo.com) wrote: > > > Hi, > > > > I'm fairly new to elasticsearch, so sorry if this is a trivial question. > > > > I'm running a typical logstash-redis-elasticsearch system to capture all > > my > > logs(around 500 GB/day). To my knowledge elasticsearch queries every shard > > in an index and aggregates the results, but due to the volume of logs per > > day and the response times needed, I want to query only few shards which > > of > > course should be decided on some "tag" in the message. So I'm looking at a > > way to allocate data to shards based on some tags and query only relevant > > shards based on the tags. Any leads, references or solutions on how to > > achieve this ? > > > > I've already looked at shard allocation filtering but that doesn't cater > > this specific requirement. > > > > Thanks, > > Aryan > > > > -- > > View this message in context: > > [http://elasticsearch-users.115913.n3.nabble.com/Query-and-allocate-data-to-shards-based-on-tags-tp4037420.html](http://elasticsearch-users.115913.n3.nabble.com/Query-and-allocate-data-to-shards-based-on-tags-tp4037420.html) > > Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:27am UTC](https://discuss.elastic.co/t/query-and-allocate-data-to-shards-based-on-tags/12632/4 "2017-07-06T02:27:15Z")