# Node indexing data locality **URL:** \ **Category:** Elasticsearch\ **Created:** [January 28, 2013, 4:16pm UTC](https://discuss.elastic.co/t/node-indexing-data-locality/10526 "2013-01-28T16:16:15Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Hadar\_Rottenberg](https://avatars.discourse-cdn.com/v4/letter/h/49beb7/32.png) [@Hadar\_Rottenberg](https://discuss.elastic.co/u/Hadar_Rottenberg)\ **Post date:** [January 28, 2013, 4:16pm UTC](https://discuss.elastic.co/t/node-indexing-data-locality/10526/1 "2013-01-28T16:16:15Z")
I have a dataset I would like to index without any specific document shard routing importance. The dataset is TBs and I find the network trips of redstributing the documents across the cluster very wasteful. What I would like to do is that each node will index its own local data. Is it possible to do that? perhaps programatically? -- ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [January 28, 2013, 6:22pm UTC](https://discuss.elastic.co/t/node-indexing-data-locality/10526/2 "2013-01-28T18:22:02Z")
Have you tried bulk indexing? It reduces network cycles effectively. Note, also the node client is using the network to communicate with the cluster nodes. Usually, there is no advantage in indexing data only to the local node, since all documents should be distributed over the data nodes by the sharding algorithm. You can enforce shard locality of all docs indexed by using the same \_route parameter value, and assign shards and nodes statically. But it's tedious to implement such things, many cool Elasticsaerch features are disabled then. Jörg Am 28.01.13 17:16, schrieb Hadar Rottenberg: > I have a dataset I would like to index without any specific document > shard routing importance. > The dataset is TBs and I find the network trips of redstributing the > documents across the cluster very wasteful. > What I would like to do is that each node will index its own local data. > Is it possible to do that? perhaps programatically? > > -- -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group, send email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![tarang\_dawer](https://avatars.discourse-cdn.com/v4/letter/t/c0e974/32.png) [@tarang\_dawer](https://discuss.elastic.co/u/tarang_dawer)\ **Post date:** [January 29, 2013, 2:14pm UTC](https://discuss.elastic.co/t/node-indexing-data-locality/10526/3 "2013-01-29T14:14:29Z")
how can i use the \_route parameter and use , say 3(same index , but distributed) out of my 20 (total for multiple indexes) shards for indexing ? i mean does the \_routing parameter supports wildcards ? otherwise , data'll just keep going to a single shard . On Monday, January 28, 2013 11:52:02 PM UTC+5:30, Jörg Prante wrote: > Have you tried bulk indexing? It reduces network cycles effectively. > Note, also the node client is using the network to communicate with the > cluster nodes. > Usually, there is no advantage in indexing data only to the local node, > since all documents should be distributed over the data nodes by the > sharding algorithm. You can enforce shard locality of all docs indexed > by using the same \_route parameter value, and assign shards and nodes > statically. But it's tedious to implement such things, many cool > Elasticsaerch features are disabled then. > > Jörg > > Am 28.01.13 17:16, schrieb Hadar Rottenberg: > > > I have a dataset I would like to index without any specific document > > shard routing importance. > > The dataset is TBs and I find the network trips of redstributing the > > documents across the cluster very wasteful. > > What I would like to do is that each node will index its own local data. > > Is it possible to do that? perhaps programatically? > > > > -- -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [January 29, 2013, 3:08pm UTC](https://discuss.elastic.co/t/node-indexing-data-locality/10526/4 "2013-01-29T15:08:24Z")
Exactly, the \_route is for indexing to one shard only. You can't cut down indexing to single node only, because the primary shards are usually distributed over several nodes. Jörg Am 29.01.13 15:14, schrieb tarang dawer: > how can i use the \_route parameter and use , say 3(same index , but > distributed) out of my 20 (total for multiple indexes) shards for > indexing ? > i mean does the \_routing parameter supports wildcards ? > otherwise , data'll just keep going to a single shard . -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![tarang\_dawer](https://avatars.discourse-cdn.com/v4/letter/t/c0e974/32.png) [@tarang\_dawer](https://discuss.elastic.co/u/tarang_dawer)\ **Post date:** [January 29, 2013, 3:15pm UTC](https://discuss.elastic.co/t/node-indexing-data-locality/10526/5 "2013-01-29T15:15:55Z")
not to same node , but even if it can do up to only those 3 primary nodes(of same index) which are distributed . is that possible in some way ? like if i can specify the routing parameter as index\* (\* corresponding to index1 ,index2, index3 ) all on separate nodes ? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [January 30, 2013, 9:13am UTC](https://discuss.elastic.co/t/node-indexing-data-locality/10526/6 "2013-01-30T09:13:51Z")
On Tue, 2013-01-29 at 20:45 +0530, tarang dawer wrote: > not to same node , but even if it can do up to only those 3 primary > nodes(of same index) which are distributed . is that possible in some > way ? like if i can specify the routing parameter as index\* (\* > corresponding to index1 ,index2, index3 ) all on separate nodes ? You could use multiple indices, and use "awareness" > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. to control which index is on which box. Then you can just index to index\_1 from box\_1, etc You can always run queries across all indices: ``` GET /index_1,index_2/_search ``` clint -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:54am UTC](https://discuss.elastic.co/t/node-indexing-data-locality/10526/7 "2017-07-06T02:54:02Z")