# Iterating over all data **URL:** \ **Category:** Elasticsearch\ **Created:** [September 14, 2013, 10:03pm UTC](https://discuss.elastic.co/t/iterating-over-all-data/13610 "2013-09-14T22:03:29Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![benmccann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/benmccann/32/420_2.png) [@benmccann](https://discuss.elastic.co/u/benmccann)\ **Post date:** [September 14, 2013, 10:03pm UTC](https://discuss.elastic.co/t/iterating-over-all-data/13610/1 "2013-09-14T22:03:29Z")
Hi, Is there a good way to iterate over all data for an ElasticSearch type? Is there any support for doing a map reduce against a large ES deployment? Thanks, Ben -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![drewr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/drewr/32/7803_2.png) [@drewr](https://discuss.elastic.co/u/drewr)\ **Post date:** [September 15, 2013, 3:54am UTC](https://discuss.elastic.co/t/iterating-over-all-data/13610/2 "2013-09-15T03:54:26Z")
Ben McCann wrote: > Is there a good way to iterate over all data for an > Elasticsearch type? Check out the scan search. > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. > Is there any support for doing a map reduce against a large ES > deployment? Can you be more specific? Do you want to submit some kind of batch job? You might look at the scripting support which can support arbitrary operations on your data. It's used in various places through the API. > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. Drew -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Ben\_McCann\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ben_mccann_2/32/1097_2.png) [@Ben\_McCann\_2](https://discuss.elastic.co/u/Ben_McCann_2)\ **Post date:** [September 15, 2013, 4:00am UTC](https://discuss.elastic.co/t/iterating-over-all-data/13610/3 "2013-09-15T04:00:06Z")
Thanks for the pointers. Yes, I'm looking to do a batch job to reprocess all of the records in our index. Any tips on when should we prefer scan search over scripting and vice versa for this type of operation? Thanks for the help! -Ben On Sat, Sep 14, 2013 at 8:54 PM, Drew Raines [aaraines@gmail.com](mailto:aaraines@gmail.com) wrote: > Ben McCann wrote: > > Is there a good way to iterate over all data for an Elasticsearch type? > > > > > Check out the scan search. > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/ **guide/reference/api/search/** search-type/)[http://www.elasticsearch.org/guide/reference/api/search/search-type/](http://www.elasticsearch.org/guide/reference/api/search/search-type/) > > Is there any support for doing a map reduce against a large ES deployment? > > > > > Can you be more specific? Do you want to submit some kind of batch job? > You might look at the scripting support which can support arbitrary > operations on your data. It's used in various places through the API. > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/ **guide/reference/modules/** scripting/)[http://www.elasticsearch.org/guide/reference/modules/scripting/](http://www.elasticsearch.org/guide/reference/modules/scripting/) > > Drew > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit [https://groups.google.com/d/](https://groups.google.com/d/)\*\* > topic/elasticsearch/\*\*En7J0T7RZ9U/unsubscribe[https://groups.google.com/d/topic/elasticsearch/En7J0T7RZ9U/unsubscribe](https://groups.google.com/d/topic/elasticsearch/En7J0T7RZ9U/unsubscribe) > . > To unsubscribe from this group and all its topics, send an email to > elasticsearch+unsubscribe@\*\*[googlegroups.com](http://googlegroups.com)[elasticsearch%2Bunsubscribe@googlegroups.com](mailto:elasticsearch%2Bunsubscribe@googlegroups.com) > . > For more options, visit [https://groups.google.com/\*\*groups/opt\_out](https://groups.google.com/**groups/opt_out)[https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out) > . -- > **[Ben McCann on about.me](https://about.me/benmccann)** > > I am a Founder, Entrepreneur, and Investor in Mountain View, California. Visit my website. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![drewr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/drewr/32/7803_2.png) [@drewr](https://discuss.elastic.co/u/drewr)\ **Post date:** [September 15, 2013, 4:13am UTC](https://discuss.elastic.co/t/iterating-over-all-data/13610/4 "2013-09-15T04:13:26Z")
Ben McCann wrote: > Thanks for the pointers. Yes, I'm looking to do a batch job to > reprocess all of the records in our index. Any tips on when > should we prefer scan search over scripting and vice versa for > this type of operation? The best you can do inside of ES is with the update API, possibly using a script, which can save you some bandwidth transferring your whole documents. [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/update/) If it turns out you need client-side processing, use a scan search to retrieve the docs and the bulk API to index the newly transformed docs. It's surprisingly efficient! [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/bulk/) Drew -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:16am UTC](https://discuss.elastic.co/t/iterating-over-all-data/13610/5 "2017-07-06T02:16:29Z")