I have an Elastic Index which has 100 million documents inside it and I want to understand whats the efficient way of writing a python script to filter values and then transfer the filtered values to a SQL storage ?
Hey @Monkey_D_Luffy1 and welcome to the community!
Using elasticsearch-py, you can perform your search with filters. Since you may have many documents, you will want to use search_after in order to paginate through them.
With each response, you can format your docs and write them into your preferred database.