"index.lifecycle.rollover_alias does not point to index

Elastic Stack Logstash
1

Please help:

logstash output:

I have a index template for logstash, which i defined in logstash output:

output {
    elasticsearch {
        hosts => ["https://elasticsearchurl.domain.com:9200"]
        user => 'logstash_internal'
        password => "${es_pwd}"
        ssl => true
        cacert => '/path/to/ca-bundle.trust.crt'
        sniffing => false
        index => "log-%{+yyyy.MM.dd}"
    }
}

I have a policy which is:

PUT _ilm/policy/log_policy   
{
  "policy": {                       
    "phases": {
      "hot": {                      
        "actions": {
          "rollover": {             
            "max_docs":  20,
            "max_age": "1h"
          }
        }
      },
      "delete": {
        "min_age": "1h",           
        "actions": {
          "delete": {}              
        }
      }
    }
  }
}

and an index template, which is assigned to policy and has a pattern to get logstash data, and rollover_alias is log

PUT _template/log_template
{
  "index_patterns": ["log-*"],
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 1,
    "index.lifecycle.name": "log_policy",
    "index.lifecycle.rollover_alias": "log"
  }
}

As per document, I created the bellow index manually

PUT log-000001
{
  "aliases": {
    "log": {
      "is_write_index": true
    }
  }
}

But I get the following error, during roll over action:

"index.lifecycle.rollover_alias [log] does not point to index [log-2019.07.27]",

Please can you help?

2

I believe you should be writing to the alias. So if you have an alias 'log' then your elasticsearch output should have

index => "log"
3

but if I change the index to "log", how can I have my template in Elasticsearch? and how can I have indices with the following pattern?
log-%{+yyyy.MM.dd}

4

If you want to have one index each day, what are hoping ILM will do for you?

5

The reason for ILM for me it to roll over the indices which are for example having more than specific number of docs to another index with fewer shards, and after a few months, I want to delete them... However, I need to keep each day log into a separate index.
If you see also this:
https://discuss.elastic.co/t/index-lifecycle-dilemma/177598
he wants to do the same thing, but for me it does not work.

6

I cannot reconcile rolling over on anything other that age with keeping each day in a separate index. Have you looked at using date math?

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.