# Forced Shardawareness question **URL:** \ **Category:** Elasticsearch\ **Created:** [February 2, 2018, 9:30am UTC](https://discuss.elastic.co/t/forced-shardawareness-question/118195 "2018-02-02T09:30:46Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![ludovic\_jerome](https://avatars.discourse-cdn.com/v4/letter/l/bc79bd/32.png) [@ludovic\_jerome](https://discuss.elastic.co/u/ludovic_jerome)\ **Post date:** [February 2, 2018, 9:30am UTC](https://discuss.elastic.co/t/forced-shardawareness-question/118195/1 "2018-02-02T09:30:46Z")
Hello, I am testing the forced shard awareness configuration following this section [https://www.elastic.co/guide/en/elasticsearch/reference/current/allocation-awareness.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/allocation-awareness.html). Please see below my setup: - 3 dedicated master node - 4 dedicated data node - X-pack enabled - elasticsearch version 6.1 Master configuration: > ``` > #################### > > ``` ``` ## Shard Awareness ## #################### "cluster.routing.allocation.awareness.force.%{location}.values": OVHSG,OVHRB cluster.routing.allocation.awareness.attributes: "%{location} ``` Data configuration: - On 2 data nodes > ``` > "node.attr.%{location}": 'OVHSG' > > ``` - On the remaining data nodes: > ``` > "node.attr.%{location}": 'OVHRB' > > ``` When I setup this configuration, all the existing indexes are correctly reallocated on the correct zones 'OVHSG' and 'OVHDB', but all new timebased indexes are not included. Please see below: For the replica zone: ![chrome_2018-02-02_10-22-48](https://us1.discourse-cdn.com/elastic/original/3X/e/a/ea3017e33dabaff1947158e4f51711f6630ca251.png) For the primary zone: ![chrome_2018-02-02_10-27-59](https://us1.discourse-cdn.com/elastic/original/3X/8/e/8e3d62bfed54206e91c1322feba31f398a03ff4d.png) Maybe I miss understand the forced awareness meaning but how can I forced than new indexes are correctly allocated to the correct zone? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [February 2, 2018, 9:35am UTC](https://discuss.elastic.co/t/forced-shardawareness-question/118195/2 "2018-02-02T09:35:03Z")
This is the mechanism we use when implementing hot/warm architecture, so [the example in this blog post](https://www.elastic.co/blog/hot-warm-architecture-in-elasticsearch-5-x) may help. I suspect you may have missed updating the index template used for new indices with the appropriate settings. ---
**Author:** ![ludovic\_jerome](https://avatars.discourse-cdn.com/v4/letter/l/bc79bd/32.png) [@ludovic\_jerome](https://discuss.elastic.co/u/ludovic_jerome)\ **Post date:** [February 2, 2018, 8:37pm UTC](https://discuss.elastic.co/t/forced-shardawareness-question/118195/3 "2018-02-02T20:37:23Z")
Hello Christian, thank you for your quick reply. I will work on it and give you my feedback. ---
**Author:** ![ludovic\_jerome](https://avatars.discourse-cdn.com/v4/letter/l/bc79bd/32.png) [@ludovic\_jerome](https://discuss.elastic.co/u/ludovic_jerome)\ **Post date:** [February 5, 2018, 5:32pm UTC](https://discuss.elastic.co/t/forced-shardawareness-question/118195/4 "2018-02-05T17:32:11Z")
Hello, Unfortunatelly I don't succeed to make it work. My case is not exactly like the hot/warm architecture because I want to split permanently each indexes into 2 zones at the index creation. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [February 5, 2018, 5:59pm UTC](https://discuss.elastic.co/t/forced-shardawareness-question/118195/5 "2018-02-05T17:59:55Z")
> [@ludovic\_jerome](#): > > %{location} Why do you have this in your config? What is it you are trying to achieve? ---
**Author:** ![ludovic\_jerome](https://avatars.discourse-cdn.com/v4/letter/l/bc79bd/32.png) [@ludovic\_jerome](https://discuss.elastic.co/u/ludovic_jerome)\ **Post date:** [February 6, 2018, 10:20am UTC](https://discuss.elastic.co/t/forced-shardawareness-question/118195/6 "2018-02-06T10:20:07Z")
Hello, %{location} is puppet variable, it resolves the cluster datacenter location in my case it will be "ovh". What I want is to test the forced shard allocation on my cluster, the objective is to control which nodes will hold a specific type of shards. I changed my filebeat template with the following lines: > { > "filebeat": { > "order": 1, > "index\_patterns": [ > "filebeat-\*" > ], > "settings": { > "index": { > "routing": { > "allocation": { > "require": { > "ovh": "OVHRB,OVHSG" > } > } > }, The result is cluster in red state and all new logs are rejeted: > [2018-02-06T11:08:14,095][INFO][logstash.outputs.elasticsearch] retrying failed action with response code: 503 ({"type"=\>"unavailable\_shards\_exception", "reason"=\>"[filebeat-2018.02.06][2] primary shard is not active Timeout: [1m], request: [BulkShardRequest [[filebeat-2018.02.06][2]] containing [index {[filebeat-2018.02.06][doc][WP-VamEBjUhj3xZVcHKp], source[{"@timestamp":"2018-02-06T10:00:09.951Z","offset":63643,"@version":"1","beat":{"name":"frovhlogstash01.talentsoft.local","hostname":"frovhlogstash01.talentsoft.local","version":"6.0.0"},"host":"frovhlogstash01.talentsoft.local","prospector":{"type":"log"},"source":"/var/log/auth.log","message":"Feb 6 11:00:01 frovhlogstash01 CRON[1830]: pam\_unix(cron:session): session opened for user root by (uid=0)","fields":{"type":"authlog"},"tags":["beats\_input\_codec\_plain\_applied"]}]}]]"}) > [2018-02-06T11:08:14,096][INFO][logstash.outputs.elasticsearch] retrying failed action with response code: 503 ({"type"=\>"unavailable\_shards\_exception", "reason"=\>"[filebeat-2018.02.06][1] primary shard is not active Timeout: [1m], request: [BulkShardRequest [[filebeat-2018.02.06][1]] containing [index {[filebeat-2018.02.06][doc][Wf-VamEBjUhj3xZVcHKp], source[{"@timestamp":"2018-02-06T10:00:09.951Z","offset":63740,"@version":"1","beat":{"name":"frovhlogstash01.talentsoft.local","hostname":"frovhlogstash01.talentsoft.local","version":"6.0.0"},"host":"frovhlogstash01.talentsoft.local","prospector":{"type":"log"},"source":"/var/log/auth.log","message":"Feb 6 11:00:03 frovhlogstash01 CRON[1830]: pam\_unix(cron:session): session closed for user root","fields":{"type":"authlog"},"tags":["beats\_input\_codec\_plain\_applied"]}]}]]"}) > [2018-02-06T11:08:14,096][INFO][logstash.outputs.elasticsearch] retrying failed action with response code: 503 ({"type"=\>"unavailable\_shards\_exception", "reason"=\>"[filebeat-2018.02.06][3] primary shard is not active Timeout: [1m], request: [BulkShardRequest [[filebeat-2018.02.06][3]] containing [index {[filebeat-2018.02.06][doc][Wv-VamEBjUhj3xZVcHKp], source[{"@timestamp":"2018-02-06T10:00:09.952Z","offset":160446,"@version":"1","beat":{"name":"frovhlogstash01.talentsoft.local","hostname":"frovhlogstash01.talentsoft.local","version":"6.0.0"},"host":"frovhlogstash01.talentsoft.local","prospector":{"type":"log"},"source":"/var/log/syslog","message":"Feb 6 11:00:01 frovhlogstash01 CRON[1831]: (root) CMD (puppet facts --render-as yaml |sed 's#!ruby/object:Puppet::Node::Facts##g' \>/etc/puppetlabs/mcollective/facts.yaml > 2\>&1)","fields":{"type":"syslog"},"tags":["beats\_input\_codec\_plain\_applied"]}]}]]"}) On the master: I have deleted the index filebeat-2018-02-06 and create it back with the new template settings > [2018-02-06T11:07:06,784][INFO][o.e.c.m.MetaDataDeleteIndexService] [frovhesmaster02.talentsoft.cloud] [filebeat-2018.02.06/\_GPlBhMVRR6XHvfwMtQCXA] deleting index > [2018-02-06T11:07:09,085][INFO][o.e.c.m.MetaDataCreateIndexService] [frovhesmaster02.talentsoft.cloud] [filebeat-2018.02.06] creating index, cause [api], templates [filebeat], shards [5]/[1], mappings [doc] > [2018-02-06T11:07:09,208][INFO][o.e.c.r.a.AllocationService] [frovhesmaster02.talentsoft.cloud] Cluster health status changed from [YELLOW] to [RED] (reason: [index [filebeat-2018.02.06] created]). When I changed the config to the line below, it's works but only for one type of shard. I can't set 2 attributes to the routing allocation, so how can the index templates shoud be configured? > "routing.allocation.require.ovh": "OVHRB" ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [March 6, 2018, 10:31am UTC](https://discuss.elastic.co/t/forced-shardawareness-question/118195/7 "2018-03-06T10:31:54Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.