Aliases API : error deleting index

[quentin.renoux]

Hi everyone,

TL;DR : the _aliases API throw error trying to remove index saying it doesn't exist but it does.

I'm using the aliases API to swap two indices behind an alias in a single atomic operation. I'm following the example provided in the "Multiple Actions" section of the Aliases documentation.

In addition to swapping the indices, I would like to delete the index removed from the alias, still in a "single atomic operation" fashion using the remove_index action.

Here is the query I came up with :

POST _aliases
{
  "actions": [
    {
      "remove": {
        "index": "backing_index-1",
        "alias": "testalias"
      }
    },
    {
      "add": {
        "index": "backing_index-2",
        "alias": "testalias"
      }
    },
    {
      "remove_index": {
        "index": "backing_index-1"
      }
    }
  ]
}

The first two actions are similar to those presented in the documentation.

However, I got this error after executing the query :

{
  "error": {
    "root_cause": [
      {
        "type": "index_not_found_exception",
        "reason": "no such index [backing_index-1]",
        "index_uuid": "_na_",
        "index": "backing_index-1"
      }
    ],
    "type": "index_not_found_exception",
    "reason": "no such index [backing_index-1]",
    "index_uuid": "_na_",
    "index": "backing_index-1"
  },
  "status": 404
}

This error is caused by the last action (remove_index). I've tried the same query without it and it works (but doesn't delete the old index, as excepted).

Here are all the steps you can follow to try to reproduce the error :

## Creating indices
POST backing_index-1/_doc/1
{"value": "The query reached backing_index-1"}

POST backing_index-2/_doc/1
{"value": "The query reached backing_index-2"}

## Creating the alias
POST _aliases
{
  "actions": [
    {
      "add": {
        "index": "backing_index-1",
        "alias": "testalias"
      }
    }
  ]
}

## This should return "The query reached backing_index-1"
GET testalias/_doc/1?filter_path=_source.value

## Swapping indices and deleting the old one
POST _aliases
{
  "actions": [
    {
      "remove": {
        "index": "backing_index-1",
        "alias": "testalias"
      }
    },
    {
      "add": {
        "index": "backing_index-2",
        "alias": "testalias"
      }
    },
    {
      "remove_index": {
        "index": "backing_index-1"
      }
    }
  ]
}

The workaround I've found is doing this instead, but, as you can see, the "single atomic operation" is completely lost here :

## Swaping indices
POST _aliases
{
  "actions": [
    {
      "remove": {
        "index": "backing_index-1",
        "alias": "testalias"
      }
    },
    {
      "add": {
        "index": "backing_index-2",
        "alias": "testalias"
      }
    }
  ]
}

## Deleting the old one
POST _aliases
{
  "actions": [
    {
      "remove_index": {
        "index": "backing_index-1"
      }
    }
  ]
}

Is there something I'm missing ?

I'm running Elasticsearch 8.6.0 on premise by the way.

Thanks for reading !

[Wave (Andrew Sullivan)]

Hi @quentin.renoux,
Thanks for your well written question. I see the behavior you are seeing and have a solution and a guess of what is happening. The following should work and give you the behavior you are seeking:

POST _aliases
{
  "actions": [
    {
      "add": {
        "index": "backing_index-2",
        "alias": "testalias"
      }
    },
    {
      "remove_index": {
        "index": "backing_index-1"
      }
    }
  ]
}

I like to think of atomic operations as "if one thing fails then everything gets undone". Furthermore I don't think we can assume the actions are happening as written top to bottom, since they are atomic and either all work or fail order shouldn't really matter. What I think is happening with the original "remove-add-remove_index" command is backing_index-2 is added, backing_index-1 is deleted and then "remove" tries to remove the alias but at that point it has already been deleted. That is where the index not found status is coming from. Since that fails everything gets rolled back.

remove_index by it's nature will also remove any alias assigned to it so including remove is not needed and I think is the true culprit in this case. I think the documentation could be a little clearer on this distinction. It's worded as: "For the add and remove_index actions, wildcard patterns that match both data streams and indices return an error." It makes it sound like only wildcard patterns that match will return an error, but logically having the same index name (as in your case) should also return an error which it does. I hope I'm explaining this clearly, and enjoyed researching this issue.

[quentin.renoux]

Hi @Wave,

Thank you for your answer.

I only thought "single atomic operation" as "only one request sent to the API", but now I see the "if one thing fails then everything gets undone" behavior and that's a fortunate side effect.

Because the JSON array notation keeps elements in order and the actions object of the query is an array, I thought the order of actions mattered, but it seems the "single atomic operation" context doesn't really take that into consideration and actions can occur in any order.

I share your opinion on the wording of the documentation, as well written it is, it would benefit from those little clarifications.

Your answer was really clear, thank you again for your time.