Rollover Index issue with Elasticsearch 6.2

Elastic Stack Elasticsearch
1

Hi guys, I am having an issue with rolling over indices.

So, We were trying a rollover indice with our newly setup cluster with Elasticsearch 6.2

When we are trying to rollover the indice, It gives the following error.

{
  "error": {
    "root_cause": [
      {
        "type": "illegal_argument_exception",
        "reason": "Rollover alias [active-fusion-logs] can point to multiple indices, found duplicated alias [[search-fusion-logs, active-fusion-logs]] in index template [fusion-logs]"
      }
    ],
    "type": "illegal_argument_exception",
    "reason": "Rollover alias [active-fusion-logs] can point to multiple indices, found duplicated alias [[search-fusion-logs, active-fusion-logs]] in index template [fusion-logs]"
  },
  "status": 400
}

Please find details below of the template that we are having and steps that I used. This can be fairly used to reproduce the issue.

Template name : fusion-logs

PUT _template/fusion-logs
{
  "template": "fusion-logs-*",
  "settings": {
    "number_of_shards": 2,
    "number_of_replicas": 1,
    "routing.allocation.include.box_type": "hot"
  },
  "aliases": {
    "active-fusion-logs": {},
    "search-fusion-logs": {}
  },
  "mappings": {
    "logs": {
      "properties": {
        "host": {
          "type": "keyword"
        },
        "job_id": {
          "type": "keyword"
        },
        "job_result": {
          "type": "keyword"
        }
      }
    }
  }
}

We inserted a 1000 documents in the above active-fusion-logs index and then used the following to roll over the index

POST active-fusion-logs/_rollover
{
  "conditions": {
    "max_docs":   1000
  }
}

The above API gives us an error when we are trying to rollover

Some other info about the cluster.

  1. There is no other index other than the above index.
  2. active-fusion-logs is aliased to just one write index
  3. search-fusion-logs is aliased to multiple indexes

Also, I had tried the same thing with Elasticsearch 5.3.2 and it worked as expected without the error.

2

The thing is that your write alias can indeed point by definition to multiple indices here.

When you rollover the alias, it will create another index with the same prefix.
So at some point you will have:

  • the old index with the write alias on it
  • the new index with the write alias on it (because it's in the template)

This can lead to inconsistency.

I'm not surprised by this behavior then which protects you from that.

My 2 cents

1 Like
3

Thanks for the quick response.

I was referring to the following article.

Please correct me if I am wrong, but whenever we roll an index, isn't the write alias shifted from the old indice to the new indice that is rolled over?

The above works perfectly fine with Elasticsearch 5.3.2. When I rollover, the write alias is shifted automatically to the new rolled index.

If this is not the case with Elasticsearch 6.2, could you please help me with what am I missing here? How do I rollover?

Thanks in advance! :slight_smile:

4

That's true.

What is happening here "I think" is that your template sets it as well. Practically you should write your template as:

PUT _template/fusion-logs
{
  "template": "fusion-logs-*",
  "settings": {
    "number_of_shards": 2,
    "number_of_replicas": 1,
    "routing.allocation.include.box_type": "hot"
  },
  "aliases": {
    "search-fusion-logs": {}
  },
  "mappings": {
    "logs": {
      "properties": {
        "host": {
          "type": "keyword"
        },
        "job_id": {
          "type": "keyword"
        },
        "job_result": {
          "type": "keyword"
        }
      }
    }
  }
}
5

Just upgraded to 6.2.3 myself, and this issue is hitting me as well. Am I understanding correctly that prefixes are effectively the same as aliases for the purposes of the rollover API as of 6.2.x?

6

@dadoonet has explained this really well. I would like to add more information here. We fail a rollover request in v6.2.0 if the rollover alias found in the index templates. Having the same rollover alias in index templates can cause the rollover alias point to multiple indices. Rollover action consists of two separate steps: (1) Create a new index and wait for that index to be ready; (2) Updates the alias to point to the new index. If the index template's aliases contain the rollover alias, the rollover alias will point to two indices between step 1 and step 2.

CL: https://github.com/elastic/elasticsearch/pull/28110

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.