Delete_by_query not deleting the indices

hi,

i have a python program which needs to delete the indices after 20 days of creation.

i have used delete_by_query but when i run the program, there is no error but also the indices are not getting deleted.
can someone help me with this.

im running AWS based Elasticsearch version 6.4.5

below is the code snippet -

import datetime
from datetime import date
from elasticsearch import Elasticsearch
import logging
import sys

logging.basicConfig(level=logging.INFO)

class ESUtility(object):
    def __init__(self):
        """
            Initializer
        """
        try:
            #please provide host and port applicable
            HOST = "abc.com"
            PORT = 80
            self.__ES_OBJ__ = Elasticsearch(host=HOST, port=PORT)
            logging.info("Connection established successfully")
        except Exception as e:
            error_message = "Exception while establishing connection to database" \
                        "{error}".format(error=str(e))
            logging.info(error_message)

    def delete_by_query(self,index_name,query):
        try:
            result = self.__ES_OBJ__.delete_by_query(index=index_name, body=query)
            #result = query
            return result
        except Exception as e:
            error_message = "Error while updating data" \
                        "{error}".format(error=str(e))
            logging.info(error_message)


class AutoCleanUp:
    @staticmethod
    def cleanup(no_of_days):
        try:
            tod = date.today()
            date_difference = datetime.timedelta(days=int(no_of_days))
            target_date = tod - date_difference
            #Please provide index name
            index_name = "test-logs-*"
            delete_query = {
                          "query": {
                            "bool": {
                              "must": [
                                {
                                  "range": {
                                    "date": {
                                      "lt": target_date
                                    }
                                  }
                                }
                              ]
                            }
                          }
                        }
            ESUtility().delete_by_query(index_name=index_name,
                                    query=delete_query)
            return ("Deleted Successfully")
        except Exception as e:
            logging.exception(str(e))
            raise Exception("Error while deleting data")

if __name__ == "__main__":
    AutoCleanUp.cleanup(sys.argv[1])

The delete_by_query API does not remove indices, only documents.

The best way to delete indices after a specified amount of time is using ILM, but since you are using the AWS Elasticsearch, and a pretty old version, you will need to use other tool, maybe curator can help in your case.

1 Like

Thanks for the suggestion on using curator but i would also like to know which API needs to be used for deleting indices after certain number of days of index creation.

There isn't an API to do that.

In newer versions this is done using the ILM, Index Lifecycle Management, but this does not exist in the version and distribution that you are using, so you will need to use curator or build an script to do that.

You could write an script that would check the index creation date of the index and delete it if it matches your requirements.

6.4 is EOL and no longer supported, you should upgrade.

Elasticsearch version 6.4.5 is EOL and no longer supported. Please upgrade ASAP.

(This is an automated response from your friendly Elastic bot. Please report this post if you have any suggestions or concerns :elasticheart: )

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.