Default shard allocation (where new shards are created)

Elastic Stack Elasticsearch
1

My current setup is with 10 nodes with ample space on spinning disks, and
20 nodes with smaller SSD disks.
I would like my workflow to be that all data is initially indexed on the
SSD nodes, after 10 days is reallocated to the spinning disks, after a
further 10 days the index is closed, and after a further 70 days the
indexes are deleted.
The curator is great for moving them to the spinning disks, but what I am
not sure about is how to define that initially all shards (primary and
replica) of an index should be created on the ssd nodes.
The spinning disk nodes are tagged:
node.disk_type: spinning
The ssd nodes are tagged
node.disk_type: ssd

To transfer after 10 days to spinning:
/usr/local/bin/curator --host es101 allocation --older-than 9 --rule
disk_type=spinning

But how do I define that the default location for all new shards should be
on disk_type:ssd ?

I have the example here
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-cluster.html which
I think could be modified like this:

curl -XPUT localhost:9200/_cluster/settings -d '{
"persistent" : {
"cluster.routing.allocation.require.disk_type" : "ssd"
}
}'

But for one this setting does not exist, and I'm not sure if this will stop
the shards being reallocated to spinning later on...

Any ideas how to implement my desired workflow?

Thank you!
-Robin-

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/18615b7c-8e99-42ee-b54d-ef06a1888181%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

Check what curator is doing with your index. Its probably fiddling with
index.routing.allocation.include and index.routing.allocation.exclude.
When you create the new index just set it pick up the ssd tag. You'll have
to make sure that curator knows how to strip that tag when the time comes
to move it to the spinning disks.

Nik

On Fri, Dec 19, 2014 at 3:36 AM, Robin Clarke robin13@gmail.com wrote:

My current setup is with 10 nodes with ample space on spinning disks, and
20 nodes with smaller SSD disks.
I would like my workflow to be that all data is initially indexed on the
SSD nodes, after 10 days is reallocated to the spinning disks, after a
further 10 days the index is closed, and after a further 70 days the
indexes are deleted.
The curator is great for moving them to the spinning disks, but what I am
not sure about is how to define that initially all shards (primary and
replica) of an index should be created on the ssd nodes.
The spinning disk nodes are tagged:
node.disk_type: spinning
The ssd nodes are tagged
node.disk_type: ssd

To transfer after 10 days to spinning:
/usr/local/bin/curator --host es101 allocation --older-than 9 --rule
disk_type=spinning

But how do I define that the default location for all new shards should be
on disk_type:ssd ?

I have the example here
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-cluster.html which
I think could be modified like this:

curl -XPUT localhost:9200/_cluster/settings -d '{
"persistent" : {
"cluster.routing.allocation.require.disk_type" : "ssd"
}
}'

But for one this setting does not exist, and I'm not sure if this will
stop the shards being reallocated to spinning later on...

Any ideas how to implement my desired workflow?

Thank you!
-Robin-

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/18615b7c-8e99-42ee-b54d-ef06a1888181%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/18615b7c-8e99-42ee-b54d-ef06a1888181%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPmjWd0sjd_VEY1sC_e5dswtBFT51M6wdsa1y%3D8kDx0zKMad6Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.