# Large amount of data and pagination

**URL:** <https://discuss.elastic.co/t/large-amount-of-data-and-pagination/140032>\
**Category:** Elasticsearch\
**Created:** [July 14, 2018, 2:19pm UTC](https://discuss.elastic.co/t/large-amount-of-data-and-pagination/140032 "2018-07-14T14:19:21Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![liban](https://avatars.discourse-cdn.com/v4/letter/l/b5e925/32.png) [@liban](https://discuss.elastic.co/u/liban)\
**Post date:** [July 14, 2018, 2:19pm UTC](https://discuss.elastic.co/t/large-amount-of-data-and-pagination/140032/1 "2018-07-14T14:19:21Z")

</div>

Hello ES community,

I am trying to delete a large amount of index data, the script i am using deletes just 10 documents from each index I’ve configured.

each index contains 10 000+ documents.  
is there a way to purge all data at the first time.

I don't want to delete and recreate them, but just purge all data

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [July 14, 2018, 2:50pm UTC](https://discuss.elastic.co/t/large-amount-of-data-and-pagination/140032/2 "2018-07-14T14:50:58Z")

</div>

The by far easiest way to delete all data is indeed to delete and recreate the indices. You can use the [delete by query API](https://www.elastic.co/guide/en/elasticsearch/reference/6.3/docs-delete-by-query.html), but this deletes individual documents and is much slower.

---

<div class="post-metadata">

**Author:** ![liban](https://avatars.discourse-cdn.com/v4/letter/l/b5e925/32.png) [@liban](https://discuss.elastic.co/u/liban)\
**Post date:** [July 14, 2018, 3:06pm UTC](https://discuss.elastic.co/t/large-amount-of-data-and-pagination/140032/3 "2018-07-14T15:06:23Z")

</div>

Hello @Christian_Dahlqvist,  
thanks for your answer.

my indexs are in a distant server and they are created by such application with such rights, so i don't want to delete them to avoid having indexs down somewhere or some rights of new created ones are wrong. by the way i am using Ansible to manage my distant ES nodes.

i am thinking in paginating and looping over shards, but i face some problems until now

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [July 14, 2018, 7:19pm UTC](https://discuss.elastic.co/t/large-amount-of-data-and-pagination/140032/4 "2018-07-14T19:19:23Z")

</div>

If you want to delete data you have the two options I listed unless you want to write your own application/script to do it.

---

<div class="post-metadata">

**Author:** ![Ritwik\_Sharma](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ritwik_sharma/32/33338_2.png) [@Ritwik\_Sharma](https://discuss.elastic.co/u/Ritwik_Sharma)\
**Post date:** [July 14, 2018, 10:15pm UTC](https://discuss.elastic.co/t/large-amount-of-data-and-pagination/140032/5 "2018-07-14T22:15:54Z")

</div>

@liban We can get 1000 records at a time at most and we can increase that by using scroll API. I am doing that for my 20,00,000 records. You can implement the same type of functionality for yourself to delete records by creating a script of your own.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 11, 2018, 10:16pm UTC](https://discuss.elastic.co/t/large-amount-of-data-and-pagination/140032/6 "2018-08-11T22:16:02Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.