Is there any way to see what caused a change in Template settings?

[cwilliams (C)]

I have a 4-node cluster running version 5.6.6 where a template setting accidentally got changed and I am wondering if there are any logs, etc. that I can use to identify what happened. The specific change was the number_of_shards value changed from 8 to 2, and number_of_replicas changed from 1 to 0. I was trying to make a persistent change to add Multi Field to the template and am guessing these other settings somehow got changed at the same time.

I had previously tried to use the API to change the Multi Field setting which worked but was not persistent across reboots, so I edited the /etc/elasticsearch/templates/logs.json file instead as this appears to be an alternate way to make this setting persistent. Unfortunately I found the shard/replication settings had changed only after another reboot, which appears to have reset the values back to the expected 8/1 so I'm not sure what the active template configuration was (I only have the logs.json config).

My current logs.json file has the following settings, and has not been changed again since the replica/shard values were first affected:

{
   "logs": {
      "template": "logs*",
      "order": 0,
      "settings": {
         "index": { 
            [...]
            "number_of_shards": "8",
            "number_of_replicas": "1", 
            [...]
            }
        }
    }
}

And if it is relevant the Multi Field configuration I added to several of the mappings was:

"fields": {
    "raw": {
      "type": "keyword"
    }
}

For reference, the settings I see on existing indices during the problem (1/7) and after it was resolved (1/9):

curl localhost:9200/logs-2020-01-07/_settings/index.number_of_replicas
{"logs-2020-01-07":{"settings":{"index":{"number_of_replicas":"0"}}}}
curl localhost:9200/logs-2020-01-07/_settings/index.number_of_shards
{"logs-2020-01-07":{"settings":{"index":{"number_of_shards":"2"}}}}

curl localhost:9200/logs-2020-01-09/_settings/index.number_of_replicas
{"logs-2020-01-09":{"settings":{"index":{"number_of_replicas":"1"}}}}
curl localhost:9200/logs-2020-01-09/_settings/index.number_of_shards
{"logs-2020-01-09":{"settings":{"index":{"number_of_shards":"8"}}}}

[Edit: formatting]

[spinscale (Alexander Reelsen)]

Hey,

there is no such thing as an audit log in the free version of Elasticsearch, I'm afraid. The security module however does have, which would have showed you, who executed a put template call, overriding the existing one.

If you want to know more, check out https://www.elastic.co/guide/en/elasticsearch/reference/7.5/enable-audit-logging.html

[cwilliams (C)]

Thank you for pointing this module out, I will definitely take a look at that going forward.