ResourceAlreadyExists Exception After Deleting Index

Background

I am using python's behave library to write feature tests for my code in linux environment. Some features have data that is stored in elastic search (no clusters, 5 shards, 1 replica, version 6.4.0). For this reason, before an elastic related feature, we find a suitable open port and create a new ES instance. Before a scenario ES index is created and filled up with data. After scenario, we delete the ES index to get into a clean state.

Test runs are triggered on host using make command -j to run tests parallel. There can be 5 parallel runs at most.

Problem

The other day my feature test runs failed and from the logs the following came to me as a surprise:

**00:42:24** [2020-05-08T00:42:24,541][INFO ][o.e.c.m.MetaDataDeleteIndexService] [sw2jEeH] [sessions/d3HbZo0tSRGX-7_WHPRCUw] deleting index
...
**00:42:37** es.indices.create(index=index, body=schema["body"]) **00:42:37** File "/usr/lib/python3/dist-packages/elasticsearch/client/utils.py", line 73, in _wrapped **00:42:37** return func(*args, params=params, **kwargs) **00:42:37** File "/usr/lib/python3/dist-packages/elasticsearch/client/indices.py", line 107, in create **00:42:37** params=params, body=body) **00:42:37** File "/usr/lib/python3/dist-packages/elasticsearch/transport.py", line 312, in perform_request **00:42:37** status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout) **00:42:37** File "/usr/lib/python3/dist-packages/elasticsearch/connection/http_urllib3.py", line 128, in perform_request **00:42:37** self._raise_error(response.status, raw_data) **00:42:37** File "/usr/lib/python3/dist-packages/elasticsearch/connection/base.py", line 125, in _raise_error **00:42:37** raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info) **00:42:37** elasticsearch.exceptions.RequestError: TransportError(400, 'resource_already_exists_exception', 'index [sessions/d3HbZo0tSRGX-7_WHPRCUw] already exists') **00:42:37** **00:42:37** exception: TransportError(400, 'resource_already_exists_exception', 'index [sessions/d3HbZo0tSRGX-7_WHPRCUw] already exists')

Notice that sessions index with d3HbZo0tSRGX-7_WHPRCUw uuid is stated by MetaDataDeleteIndexService as it is going to be deleted. Several seconds later, when creating a new index for an ES scenario fails like the index were existing.

My question is whether there can be a significant gap between 'deleting index' and the index is actually deleted? That would explain why ResourceAlreadyExists was thrown.

Thanking you in advance for any help and idea.

Are you sure the deletion of the index really completed before you tried to create it again? You must wait for a successful response from the delete-index request before you can know that the index was really deleted.

Thank you for your quick reply!

I do not check the success of the deletion (since I am invoking es.indices.delete() with ignore=[400,404]), but it is definitely a good approach that enhances code safety.

I'll add this check and see if the problem arises again.

Seems unwise to me to ignore 400 responses to the delete-index request. A 404 is legitimate, it means the index wasn't there, but a 400 means the request is malformed or otherwise failed.

1 Like