X-Pack Index Template conflict with existing templates. Inconsistent overrides

[cdekker]

Almost all index templates that get generated by installing X-Pack have order:0 and the following settings:

  "settings": {
    "index": {
      "number_of_shards": "1",
      "auto_expand_replicas": "0-1",
      "number_of_replicas": "0"
    }
  }

For our application indices, we have the following template set up:

{
  "order": 0,
  "index_patterns": [
    "*"
  ],
  "settings": {
    "index": {
      "number_of_shards": "4",
      "number_of_replicas": "2"
    }
  }
}

However, it seems that merging of these templates is done inconsistent. While any of my application indices are still created using 4 shards and 2 replicas, this template is also applied to some x-pack indices, even though they have a more specific template configured.

For example, .security-6 and .watcher-history-7-2018.08.09 have 1 shards and 1 replica, as expected. However, .monitoring-es-6-2018.08.26 and similar monitoring indices all have 4 shards, even though they have a specific template matching .monitoring-es-6-* set up.

Why does my more generic template override the specific ones? And even stranger, why does it only do so in some cases, yet the order is reversed in others.

[cdekker]

While the X-Pack plugins are enabled, I am unable to delete the X-Pack related templates or even update the order on them. Any changes are immediately overwritten.

It also looks like I cannot write exclude patterns in my application template. "index_patterns": ["*","-.*"] does not seem to work (i.e. apply to all indices, except those starting with a dot.

The only way I got this to work is to actually set the order of my application template to -1 to make sure the X-Pack templates with order 0 are applied last.

This feels pretty hacky, but right now I do not see any better way.