We have below ELK pipeline out using logstash input/output features. We are following weekly rotation using below config
logstash->logstash-aggregator->kafka->elasticasearch
output {
elasticsearch {
codec => "json"
hosts => [ "xxx:9200", "xxx:9200", "xxxi:9200" ]
index => "logstash-%{+YYYY.ww}"
document_type => "logs"
}
}
With first week of January, it is sending logs to two different indexes instead of one
lotstash-2016.53
logstash-2017.01
We are at version logstash-2.0.0 for logstash elastic-2.0.0 for elastic search