Configure deprecation log index properties

Elastic Stack Elasticsearch
1

7.16 adds a new stream/index for deprecation logging.
e.g. the first one today is .ds-.logs-deprecation.elasticsearch-default-2021.12.09-000001

However, it is created with "number_of_replicas": "1" and I only have one node, so it's always yellow. (I don't care about possible data loss.)

How to configure the replica settings for these indexes?
Via a config setting or an index template?

1 Like
2

Try:
curl -X PUT "127.0.0.1:9200/.logs-deprecation.elasticsearch-default/_settings" -H 'Content-Type: application/json' -d'{ "index" : { "number_of_replicas" : 0 } }'

1 Like
3

But does that also apply to all the future indexes as the date rolls?

4

It doesn't work with future indexes.

Let's try this one:

curl -X PUT "127.0.0.1:9200/.*/_settings" -H 'Content-Type: application/json' -d'{ "index" : { "number_of_replicas" : 0 } }'

Hopefully someone will find better solution.

5

That still doesn't work with future indexes

Is there no index template or config to control the deprecation log index?

6

I have the same problem and have posted here

Creating a template (legacy) doesn't appear to work even though it matches. Need to read up on datastream

[2022-01-22T15:30:10,295][WARN ][o.e.c.m.MetadataIndexTemplateService] [elastic] legacy template [.logs-deprecation.elasticsearch-default] has index patterns [.logs-deprecation.elasticsearch-default-*] matching patterns from existing composable templates [.deprecation-indexing-template] with patterns (.deprecation-indexing-template => [.logs-deprecation.*]); this template [.logs-deprecation.elasticsearch-default] may be ignored in favor of a composable template at index creation time
[2022-01-22T15:30:10,296][INFO ][o.e.c.m.MetadataIndexTemplateService] [`Preformatted text`elastic] adding template [.logs-deprecation.elasticsearch-default] for index patterns [.logs-deprecation.elasticsearch-default-*]
7

Upgraded to 8.0.0 and my node is still yellow, the other thread I opened has been closed

Anyone else that can help us with this?

@Toby-Elastic

Sorry, I tried to follow but unsure how to fix?

8

I found the the component template .deprecation-indexing-settings.
However, I am unable to update it.

$ curl 'http://localhost:9200/_component_template/.deprecation-indexing-settings'
{"component_templates":[{"name":".deprecation-indexing-settings","component_template":{"template":{"settings":{"index":{"lifecycle":{"name":".deprecation-indexing-ilm-policy"},"codec":"best_compression","hidden":"true","auto_expand_replicas":"0-1","query":{"default_field":["message"]}}}},"version":1,"_meta":{"managed":true,"description":"default settings for Stack deprecation logs index template installed by x-pack"}}}]}
$ curl 'http://localhost:9200/_component_template/.deprecation-indexing-settings' -X PUT -H Content-Type:application/json --data '{"template":{"settings":{"index":{"lifecycle":{"name":".deprecation-indexing-ilm-policy"},"codec":"best_compression","hidden":"true","number_of_replicas":"0","auto_expand_replicas":"0-1","query":{"default_field":["message"]}}}}}'
{"acknowledged":true}
$ curl 'http://localhost:9200/_component_template/.deprecation-indexing-settings'
{"component_templates":[{"name":".deprecation-indexing-settings","component_template":{"template":{"settings":{"index":{"lifecycle":{"name":".deprecation-indexing-ilm-policy"},"codec":"best_compression","hidden":"true","auto_expand_replicas":"0-1","query":{"default_field":["message"]}}}},"version":1,"_meta":{"managed":true,"description":"default settings for Stack deprecation logs index template installed by x-pack"}}}]}

What's going on here?

9

Thanks for that, I didn't realize component templates are separated from the old _template

When I check the settings on my server it looks to have the correct settings

curl -XGET -k -u elastic:$pass https://localhost:9200/.logs-deprecation.elasticsearch-default/_settings?pretty
{
  ".ds-.logs-deprecation.elasticsearch-default-2022.01.17-000001" : {
    "settings" : {
      "index" : {
        "lifecycle" : {
          "name" : ".deprecation-indexing-ilm-policy",
          "indexing_complete" : "true"
        },
        "codec" : "best_compression",
        "routing" : {
          "allocation" : {
            "include" : {
              "_tier_preference" : "data_hot"
            }
          }
        },
        "hidden" : "true",
        "number_of_shards" : "1",
        "auto_expand_replicas" : "0-1",
        "provided_name" : ".ds-.logs-deprecation.elasticsearch-default-2022.01.17-000001",
        "query" : {
          "default_field" : [
            "message"
          ]
        },
        "creation_date" : "1642378528109",
        "number_of_replicas" : "0",
        "uuid" : "ejvRQCjMSs6QP7MGnJsNzg",
        "version" : {
          "created" : "7160399"
        }
      }
    }
  },
10

@OrangeDog
This one worked for me

curl -XPUT -k -u elastic:$pass "https://localhost:9200/_component_template/.deprecation-indexing-settings" -H 'Content-Type: application/json' -d '{"template":{"settings":{"index":{"lifecycle":{"name":".deprecation-indexing-ilm-policy"},"codec":"best_compression","hidden":"true","number_of_replicas":"0","auto_expand_replicas":"0-1","query":{"default_field":["message"]}}}},"version":1,"_meta":{"managed":true,"description":"default settings for Stack deprecation logs index template installed by x-pack"}}}'

Note: First I've tried this one and changed the version number but it completely overwrote my settings so be careful!

curl -XPUT -k -u elastic:$pass "https://localhost:9200/_component_template/.deprecation-indexing-settings" -H 'Content-Type: application/json' -d '{"template": {"settings": {"number_of_replicas": 0}},"version":2}'

All of my data streams are yellow, would be nice if someone could show us how to change the stream defaults to zero replicas like in the past with deprecated templates

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.