5.6 -> 6.1 Upgrade logstash template woes (SOLVED)

[Chris_Stone]

After performing a 5.6 to 6.1 upgrade of my ELK stack, I noticed these errors in the logstash logs:

[2018-01-03T08:16:53,027][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"logstash-2018.01.03", :_type=>"doc", :_routing=>nil}, #<LogStash::Event:0x7b1ba573>], :response=>{"index"=>{"_index"=>"logstash-2018.01.03", "_type"=>"doc", "_id"=>nil, "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"Failed to parse mapping [_default_]: [include_in_all] is not allowed for indices created on or after version 6.0.0 as [_all] is deprecated. As a replacement, you can use an [copy_to] on mapping fields to create your own catch all field.", "caused_by"=>{"type"=>"mapper_parsing_exception", "reason"=>"[include_in_all] is not allowed for indices created on or after version 6.0.0 as [_all] is deprecated. As a replacement, you can use an [copy_to] on mapping fields to create your own catch all field."}}}}}`

I then deleted the logstash template and added a new one from here:

https://github.com/logstash-plugins/logstash-output-elasticsearch/blob/master/lib/logstash/outputs/elasticsearch/elasticsearch-template-es6x.json

I then deleted the day's logstash index and logstash seemed to create a new one. However, the cluster status still shows as red with this unassigned shards:

logstash-2018.01.03 4 p UNASSIGNED INDEX_CREATED
logstash-2018.01.03 3 p UNASSIGNED INDEX_CREATED
logstash-2018.01.03 1 p UNASSIGNED INDEX_CREATED
logstash-2018.01.03 2 p UNASSIGNED INDEX_CREATED
logstash-2018.01.03 0 p UNASSIGNED INDEX_CREATED

And logstash logs show:

[2018-01-03T09:30:34,967][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 503 ({"type"=>"unavailable_shards_exception", "reason"=>"[logstash-2018.01.03][0] primary shard is not active Timeout: [1m], request: [BulkShardRequest [[logstash-2018.01.03][0]] containing [36] requests]"})

How to remedy this?

Thanks!

[Chris_Stone]

Fixed....I just needed to reenable allocation, doh!

PUT /_cluster/settings
{
"transient" : {
"cluster.routing.allocation.enable" : "all"
}
}