Managing "is_write_index"

Elastic Stack Elasticsearch
1

Hi there,

We've created a Lambda function (python) in order to steam logs from AWS CloudWatch to our ELK stack. I'm also making use of life cycle policies to rotate the index on a specific size and delete after the specified days.

The python script creates an index in this format: cwl-xxxxxx-20201120-0001. When the index is created, I set "is_write_index : True"

However, on the next day, when creating a new index for the day with the Lambda function, I get the following error:

alias [cwl-xxxxxx] has more than one write index [cwl-xxxxxx-2020.11.20-0001,cwl-xxxxxx-2020.11.19-000002].

I understand that I can only have one write index, but how do I manage that? I thought that when setting the write index on the new index, it will be removed from the previous index?

I think I'm just missing something here. Any advice would be greatly appreciated.

2

Hi,

Have a look here: At the bottom of the page there is an example how to atomically switch the write index. Basically, you create the index without is_write_index and use the alias API to switch the write index to the new one.

POST /_aliases
{
  "actions": [
    {
      "add": {
        "index": "test",
        "alias": "alias1",
        "is_write_index": false
      }
    }, {
      "add": {
        "index": "test2",
        "alias": "alias1",
        "is_write_index": true
      }
    }
  ]
}

Best regards
Wolfram

3

If you are using ILM with rollover you should write directly to the write alias and not a specific index name. ILM will then behind the scenes roll indices as required as per the policy configuration. ILM will switch the write alias to point to the correct index.