Curator makes is possible to migrate an index to another storage programmatically, and that's very nice to keep old indices on cheap storage. But if I understand correctly, a unique ES cluster cannot handle two different storages. Hence, having small but fast storage for recent files and cheap but slow storage for old files requires building two clusters.
Am I right?
Curator makes is possible to migrate an index to another storage
programmatically, and that's very nice to keep old indices on cheap
storage. But if I understand correctly, a unique ES cluster cannot handle
two different storages. Hence, having small but fast storage for recent
files and cheap but slow storage for old files requires building two
clusters.
Am I right?
Ok, so if I understand correctly I can have a single cluster with:
machine A: fast storage (recent data)
machine B & C: slow storage (old data)
In that case, I cannot have a homogeneous cluster with both fast and slow storage on each node and I'm losing the benefit of having multiple machines when I index new data and when I search recent data. Is that correct?
Curator makes is possible to migrate an index to another storage
programmatically, and that's very nice to keep old indices on cheap
storage. But if I understand correctly, a unique ES cluster cannot handle
two different storages. Hence, having small but fast storage for recent
files and cheap but slow storage for old files requires building two
clusters.
Am I right?
You cannot have multiple data.paths on a single node/instances. You could
try running multiple instances of ES on a single physical, each pointing to
either one of your tiered pools.
But you aren't losing the benefit of multiple nodes, just the optimal use
of your storage on those physical nodes.
You could look at something like L2ARC or similar though.
Ok, so if I understand correctly I can have a single cluster with:
machine A: fast storage (recent data)
machine B & C: slow storage (old data)
In that case, I cannot have a homogeneous cluster with both fast and slow
storage on each node and I'm losing the benefit of having multiple machines
when I index new data and when I search recent data. Is that correct?
Curator makes is possible to migrate an index to another storage
programmatically, and that's very nice to keep old indices on cheap
storage. But if I understand correctly, a unique ES cluster cannot
handle
two different storages. Hence, having small but fast storage for recent
files and cheap but slow storage for old files requires building two
clusters.
Am I right?
thanks,
Patrick
--
You received this message because you are subscribed to the Google
Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
You cannot have multiple data.paths on a single node/instances. You could try running multiple instances of ES on a single physical, each pointing to either one of your tiered pools.
But you aren't losing the benefit of multiple nodes, just the optimal use of your storage on those physical nodes.
You could look at something like L2ARC or similar though.
On 15 July 2014 17:05, Patrick Proniewski elasticsearch@patpro.net wrote:
Ok, so if I understand correctly I can have a single cluster with:
machine A: fast storage (recent data)
machine B & C: slow storage (old data)
In that case, I cannot have a homogeneous cluster with both fast and slow storage on each node and I'm losing the benefit of having multiple machines when I index new data and when I search recent data. Is that correct?
Curator makes is possible to migrate an index to another storage
programmatically, and that's very nice to keep old indices on cheap
storage. But if I understand correctly, a unique ES cluster cannot handle
two different storages. Hence, having small but fast storage for recent
files and cheap but slow storage for old files requires building two
clusters.
Am I right?
You cannot have multiple data.paths on a single node/instances. You
could try running multiple instances of ES on a single physical, each
pointing to either one of your tiered pools.
But you aren't losing the benefit of multiple nodes, just the optimal
use of your storage on those physical nodes.
You could look at something like L2ARC or similar though.
On 15 July 2014 17:05, Patrick Proniewski elasticsearch@patpro.net
wrote:
Ok, so if I understand correctly I can have a single cluster with:
machine A: fast storage (recent data)
machine B & C: slow storage (old data)
In that case, I cannot have a homogeneous cluster with both fast and
slow storage on each node and I'm losing the benefit of having multiple
machines when I index new data and when I search recent data. Is that
correct?
Curator makes is possible to migrate an index to another storage
programmatically, and that's very nice to keep old indices on cheap
storage. But if I understand correctly, a unique ES cluster cannot
handle
two different storages. Hence, having small but fast storage for
recent
files and cheap but slow storage for old files requires building two
clusters.
Am I right?
thanks,
Patrick
--
You received this message because you are subscribed to the Google
Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
On Tuesday, July 15, 2014 1:20:39 AM UTC-4, Patrick Proniewski wrote:
Hello,
Curator makes is possible to migrate an index to another storage
programmatically, and that's very nice to keep old indices on cheap
storage. But if I understand correctly, a unique ES cluster cannot handle
two different storages. Hence, having small but fast storage for recent
files and cheap but slow storage for old files requires building two
clusters.
Am I right?
Not necessarily. We used the tiered storage approach in Logsene http://sematext.com/logsene/, for example, but we explicitly move older
indexes to from more expensive nodes that deal with fresh data to cheaper
nodes that host all data. It's automated, but it's not 100% done within
ES. But it's done with a single ES cluster.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
