Indexing strategy advice

Elastic Stack Elasticsearch
1

Hello all,

We're currently evaluating how we should setup our index strategy for a new
ES project.

We're expecting about 1.5 million documents a month to be indexed and we're
looking at creating Aliases with routing and filters to simulate per user
indices.

Should we just use one index for all this? I'm sure we'll be plenty fine
for a while, but I was thinking into the future and if we want to roll data
off or maybe move some of the older data into another cluster. If we did a
time series index, how does that work with aliases? If we have thousands
of aliases, at midnight of the time series rollover, do we need to update
the thousands of aliases to point to the new index or can we have those
aliases point to an index alias itself?

Also is there a way to dynamically create aliases based on field data or is
that something that needs to be done manually at this point? Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/56a99f81-73a6-4d86-9c28-b7c1092bda3b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

You'd be better off moving data into different indexes on the same cluster,
then growing that cluster.

Aliases are at an index level, so if you use time based indexes then you
need to simply add the existing alias to that index, not update the alias.
Check out

as it has good info.

On 25 November 2014 at 10:30, Justin Lintz jlintz@gmail.com wrote:

Hello all,

We're currently evaluating how we should setup our index strategy for a
new ES project.

We're expecting about 1.5 million documents a month to be indexed and
we're looking at creating Aliases with routing and filters to simulate per
user indices.

Should we just use one index for all this? I'm sure we'll be plenty fine
for a while, but I was thinking into the future and if we want to roll data
off or maybe move some of the older data into another cluster. If we did a
time series index, how does that work with aliases? If we have thousands
of aliases, at midnight of the time series rollover, do we need to update
the thousands of aliases to point to the new index or can we have those
aliases point to an index alias itself?

Also is there a way to dynamically create aliases based on field data or
is that something that needs to be done manually at this point? Thanks

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/56a99f81-73a6-4d86-9c28-b7c1092bda3b%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/56a99f81-73a6-4d86-9c28-b7c1092bda3b%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAF3ZnZkmE_bKRK4tBMuVFNkBovv%2BB%2B0WtpwuAzT3sO%2BBdsijoQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.