Create index with metadata

[pjanzen (Paul Janzen)]

Hi All,

From Logstash 6.8.0 I push data to Elasticsearch 6.8.0 using a Logstash output like this.

output {
  if "_dissectfailure" in [tags] {
    file {
      path => '/opt/logstash/logs/dissectfailure.txt'
      codec => rubydebug
    }
  }
  if "_geoip_lookup_failure" in [tags] {
    file {
      path => '/opt/logstash/logs/geoipfailure.txt'
      codec => rubydebug
    }
  }
  if [type] == 'cloned_dovecot' {
    elasticsearch {
      hosts => ["10.80.3.180", "10.80.3.181", "10.80.3.182", "10.80.3.183", "10.80.3.184", "10.80.3.185"]
      index => "clog_dovecot_longterm-%{+YYYY.MM.dd}"
      id => "reporting_output"
    }
  } else {
    elasticsearch {
      hosts => ["10.80.3.11","10.80.3.12","10.80.3.13","10.80.3.14","10.80.3.15","10.80.3.16","10.80.3.17","10.80.3.18","10.80.3.19","10.80.3.20","10.80.3.21","10.80.3.22","10.80.3.23","10.80.3.24","10.80.3.25","10.80.3.26","10.80.3.27","10.80.3.28","10.80.3.29","10.80.3.30"]
      index => "clog-%{+YYYY.MM.dd}"
      user => "elastic"
      password => "xxxx"
      template_overwrite => "false"
      id => "elastic_output"
    }
  }
}

Different data is written to different clusters (globally the version is 6.8.0). I the bottom case I get a index with name: clog-2019.08.28 but in the other case I just get the name: clog_dovecot_longterm-

I have checked the setting on both clusters and the are the same, it looks like I am missing something but I cannot seem to find out what.

Can someone please give some advice?

Thanks,
Paul.

[pjanzen (Paul Janzen)]

Elasticsearch does not provide any inside

[2019-08-29T12:00:04,788][INFO ][o.e.c.m.MetaDataCreateIndexService] [esearchr1xxxx] [clog_dovecot_longterm-] creating index, cause [auto(bulk api)], templates [clog-dovecot_longterm], shards [6]/[1], mappings [doc]
[2019-08-29T12:00:05,626][INFO ][o.e.c.r.a.AllocationService] [esearchr1xxx] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[clog_dovecot_longterm-][1]] ...]).
[2019-08-29T12:00:09,338][INFO ][o.e.c.m.MetaDataIndexTemplateService] [esearchr1xxx] adding template [clog-dovecot_longterm] for index patterns [clog_dovecot_longterm-*]