# Delete the data in Elasticsearch index based on a date/timestamp column in that index using python

**URL:** <https://discuss.elastic.co/t/delete-the-data-in-elasticsearch-index-based-on-a-date-timestamp-column-in-that-index-using-python/254775>\
**Category:** Kibana\
**Created:** [November 9, 2020, 3:18pm UTC](https://discuss.elastic.co/t/delete-the-data-in-elasticsearch-index-based-on-a-date-timestamp-column-in-that-index-using-python/254775 "2020-11-09T15:18:51Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![vidhyadhara](https://avatars.discourse-cdn.com/v4/letter/v/c0e974/32.png) [@vidhyadhara](https://discuss.elastic.co/u/vidhyadhara)\
**Post date:** [November 9, 2020, 3:18pm UTC](https://discuss.elastic.co/t/delete-the-data-in-elasticsearch-index-based-on-a-date-timestamp-column-in-that-index-using-python/254775/1 "2020-11-09T15:18:52Z")

</div>

I have several indexes which has a column named "date" or "timestamp", I want to delete only the data from that index which older than certain days/weeks. I do not want to delete the entire index.

In fact I want to do it from python, if someone is able to suggest me how to do it from kibana also its fine for now.

Thanks in advance 😀

---

<div class="post-metadata">

**Author:** ![flash1293](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/flash1293/32/41227_2.png) [@flash1293](https://discuss.elastic.co/u/flash1293)\
**Post date:** [November 9, 2020, 3:35pm UTC](https://discuss.elastic.co/t/delete-the-data-in-elasticsearch-index-based-on-a-date-timestamp-column-in-that-index-using-python/254775/2 "2020-11-09T15:35:39Z")

</div>

Kibana is mostly a tool to view your data, not to modify it, so you can't do this from within let's say Discover. Your best bet here are the "Dev Tools" ([https://www.elastic.co/guide/en/kibana/current/console-kibana.html](https://www.elastic.co/guide/en/kibana/current/console-kibana.html)) - those allow you to talk to Elasticsearch REST APIs directly, so it won't be a big jump to do it in dev tools vs Python.

To delete only certain documents, `_delete_by_query` seems like the right choice: [https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-delete-by-query.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-delete-by-query.html)

For Python in general it's recommended to use the official python client: [https://elasticsearch-py.readthedocs.io/en/7.9.1/](https://elasticsearch-py.readthedocs.io/en/7.9.1/)

---

<div class="post-metadata">

**Author:** ![wylie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wylie/32/81794_2.png) [@wylie](https://discuss.elastic.co/u/wylie)\
**Post date:** [November 9, 2020, 4:53pm UTC](https://discuss.elastic.co/t/delete-the-data-in-elasticsearch-index-based-on-a-date-timestamp-column-in-that-index-using-python/254775/3 "2020-11-09T16:53:13Z")

</div>

We also have a free feature called Index Lifecycle Management which lets you configure your indices to automatically get deleted after a certain time, details here: [https://www.elastic.co/guide/en/elasticsearch/reference/current/index-lifecycle-management.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-lifecycle-management.html)

Specifically, you can set a delete phase that can be met based on the conditions you want.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [November 10, 2020, 6:19am UTC](https://discuss.elastic.co/t/delete-the-data-in-elasticsearch-index-based-on-a-date-timestamp-column-in-that-index-using-python/254775/4 "2020-11-10T06:19:21Z")

</div>

Just to add on another comment, you are far better off using time-based indices rather than deleting data from an index based on a timestamp. It's far more efficient.

---

<div class="post-metadata">

**Author:** ![vidhyadhara](https://avatars.discourse-cdn.com/v4/letter/v/c0e974/32.png) [@vidhyadhara](https://discuss.elastic.co/u/vidhyadhara)\
**Post date:** [November 11, 2020, 6:42am UTC](https://discuss.elastic.co/t/delete-the-data-in-elasticsearch-index-based-on-a-date-timestamp-column-in-that-index-using-python/254775/5 "2020-11-11T06:42:35Z")

</div>

This served my purpose, just what I needed.  
Thank you very much 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 9, 2020, 6:42am UTC](https://discuss.elastic.co/t/delete-the-data-in-elasticsearch-index-based-on-a-date-timestamp-column-in-that-index-using-python/254775/6 "2020-12-09T06:42:37Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
