Atomically create index with single alias

Elastic Stack Elasticsearch
1

I have a time-series index. I create a new version once a day using the
format index-name-YYYY-mm-dd.

When creating the index, I assign it a constant alias of 'index-name'.
This way, clients can just refer to the 'index-name', and not have to
append the timestamp. I create the alias when I create the index so it's
an atomic operation; however I create a new index every day and am
assigning it the same alias. This is blowing up as you can't have the same
alias point to multiple indexes. I'm trying to fix this but can't find an
atomic way to:

  1. create the new index
  2. remove the alias from the old index
  3. add the alias to the new index

I can do 1/3 atomically using the index API or 2/3 atomically using the
alias API, but I can't find a way to do all three.

Any ideas?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/6322f948-6574-45ce-b859-773dbae1f1da%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

You can have an alias point to multiple indices. With time-series data,
this should be a problem since you will not have overlap between the
different indices. But I think you are correct in that there is no atomic
way to accomplish all three.

I only search against aliases, so having an index without an alias is
acceptable since I can accomplish the last 2 steps atomically without
interfering querying (some data might not be available).

Cheers,

Ivan

On Tue, Jul 29, 2014 at 11:20 AM, Matt Hughes hughes.matt@gmail.com wrote:

I have a time-series index. I create a new version once a day using the
format index-name-YYYY-mm-dd.

When creating the index, I assign it a constant alias of 'index-name'.
This way, clients can just refer to the 'index-name', and not have to
append the timestamp. I create the alias when I create the index so it's
an atomic operation; however I create a new index every day and am
assigning it the same alias. This is blowing up as you can't have the same
alias point to multiple indexes. I'm trying to fix this but can't find an
atomic way to:

  1. create the new index
  2. remove the alias from the old index
  3. add the alias to the new index

I can do 1/3 atomically using the index API or 2/3 atomically using the
alias API, but I can't find a way to do all three.

Any ideas?

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/6322f948-6574-45ce-b859-773dbae1f1da%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/6322f948-6574-45ce-b859-773dbae1f1da%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBRfb37gJ3Ytmg_kGus-uVNn9h85uvpjG-Et4QM4eZTZw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

I do an atomic alias swap all the time, but I do it after the index is
created:

curl -XPOST 'http://localhost:9200/_aliases' -d '
{
"actions" : [
{ "remove" : { "index" : "index1", "alias" : "alias" } },

    { "add" : { "index" : "index2", "alias" : "alias" } }

]

}'

Nik

On Tue, Jul 29, 2014 at 2:57 PM, Ivan Brusic ivan@brusic.com wrote:

You can have an alias point to multiple indices. With time-series data,
this should be a problem since you will not have overlap between the
different indices. But I think you are correct in that there is no atomic
way to accomplish all three.

I only search against aliases, so having an index without an alias is
acceptable since I can accomplish the last 2 steps atomically without
interfering querying (some data might not be available).

Cheers,

Ivan

On Tue, Jul 29, 2014 at 11:20 AM, Matt Hughes hughes.matt@gmail.com
wrote:

I have a time-series index. I create a new version once a day using the
format index-name-YYYY-mm-dd.

When creating the index, I assign it a constant alias of 'index-name'.
This way, clients can just refer to the 'index-name', and not have to
append the timestamp. I create the alias when I create the index so it's
an atomic operation; however I create a new index every day and am
assigning it the same alias. This is blowing up as you can't have the same
alias point to multiple indexes. I'm trying to fix this but can't find an
atomic way to:

  1. create the new index
  2. remove the alias from the old index
  3. add the alias to the new index

I can do 1/3 atomically using the index API or 2/3 atomically using the
alias API, but I can't find a way to do all three.

Any ideas?

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/6322f948-6574-45ce-b859-773dbae1f1da%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/6322f948-6574-45ce-b859-773dbae1f1da%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBRfb37gJ3Ytmg_kGus-uVNn9h85uvpjG-Et4QM4eZTZw%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBRfb37gJ3Ytmg_kGus-uVNn9h85uvpjG-Et4QM4eZTZw%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPmjWd1DZMF7QEkokC3nzoFjX8gRqUg_TXZQy6PVUjZgt9xFFA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

4

Typo: With time-series data, this should NOT be a problem

On Tue, Jul 29, 2014 at 11:57 AM, Ivan Brusic ivan@brusic.com wrote:

You can have an alias point to multiple indices. With time-series data,
this should be a problem since you will not have overlap between the
different indices. But I think you are correct in that there is no atomic
way to accomplish all three.

I only search against aliases, so having an index without an alias is
acceptable since I can accomplish the last 2 steps atomically without
interfering querying (some data might not be available).

Cheers,

Ivan

On Tue, Jul 29, 2014 at 11:20 AM, Matt Hughes hughes.matt@gmail.com
wrote:

I have a time-series index. I create a new version once a day using the
format index-name-YYYY-mm-dd.

When creating the index, I assign it a constant alias of 'index-name'.
This way, clients can just refer to the 'index-name', and not have to
append the timestamp. I create the alias when I create the index so it's
an atomic operation; however I create a new index every day and am
assigning it the same alias. This is blowing up as you can't have the same
alias point to multiple indexes. I'm trying to fix this but can't find an
atomic way to:

  1. create the new index
  2. remove the alias from the old index
  3. add the alias to the new index

I can do 1/3 atomically using the index API or 2/3 atomically using the
alias API, but I can't find a way to do all three.

Any ideas?

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/6322f948-6574-45ce-b859-773dbae1f1da%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/6322f948-6574-45ce-b859-773dbae1f1da%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBAk5nvKQvV0XjqM5QvOvD5Ruzsa_ietuup5a2A-Ks26g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.