Is adding and removing an index with a single call to the aliases api atomic?


(Frederick Cheung) #1

Hi,

When we need to rebuild an index from scratch we handle the transition
using aliases, for example we'd start off with garments-2012-04-01
having the alias garments, then we'd build garments-2012-05-01 and
when we're done we change the aliases around so that garments is now
an alias for garments-2012-05-01. The app always queries the index
using the garments aliases and so it gets a smooth transition from 1
index to another.

We use to do this by doing a delete on garments (which thus deleting
garments-2012-04-01) and then adding the garments alias to
garments-2012-05-01. While this obviously created a window in which
queries would fail, what I had forgotten about was that since some
apis automatically create the index, there was a race condition where
a garments index would be created automatically and my adding the new
alias would fail, so the app is left using the empty auto created
index.

I was considering shuffling the indexes around by removing and adding
the index in one call to the aliases api, something like

curl -XPOST 'http://localhost:9200/_aliases' -d '
{
"actions" : [
{ "remove" : { "index" : "garments-2012-04-01", "alias" :
"garments" } },
{ "add" : { "index" : "garments-2012-05-01", "alias" :
"garments" } }
]
}'

Does this avoid the pitfall I encountered previously or am I better
off just turning off action.auto_create_index ?

Thanks for any advice,

Fred


(Andrew[.:at:.]DataFeedFile.com) #2

Have you tried adding BOTH indices into a single Alias, then you can
delete one index while the other still exists.
Would that solve your problem?

On May 9, 9:10 am, Frederick Cheung frederick.che...@gmail.com
wrote:

Hi,

When we need to rebuild an index from scratch we handle the transition
using aliases, for example we'd start off with garments-2012-04-01
having the alias garments, then we'd build garments-2012-05-01 and
when we're done we change the aliases around so that garments is now
an alias for garments-2012-05-01. The app always queries the index
using the garments aliases and so it gets a smooth transition from 1
index to another.

We use to do this by doing a delete on garments (which thus deleting
garments-2012-04-01) and then adding the garments alias to
garments-2012-05-01. While this obviously created a window in which
queries would fail, what I had forgotten about was that since some
apis automatically create the index, there was a race condition where
a garments index would be created automatically and my adding the new
alias would fail, so the app is left using the empty auto created
index.

I was considering shuffling the indexes around by removing and adding
the index in one call to the aliases api, something like

curl -XPOST 'http://localhost:9200/_aliases'-d '
{
"actions" : [
{ "remove" : { "index" : "garments-2012-04-01", "alias" :
"garments" } },
{ "add" : { "index" : "garments-2012-05-01", "alias" :
"garments" } }
]

}'

Does this avoid the pitfall I encountered previously or am I better
off just turning off action.auto_create_index ?

Thanks for any advice,

Fred


(Shay Banon) #3

Yes, thats the idea of the aliases API, the "operations" that you specify
there are atomic (all or nothing). Note, teh remove simply states that the
alias will be removed, not that the index will be removed/deleted.

Also, you can set action.auto_create_index to false in the config file
which will disable automatically creating an index if it does not exists
with the index API.

On Wed, May 9, 2012 at 5:10 PM, Frederick Cheung <frederick.cheung@gmail.com

wrote:

Hi,

When we need to rebuild an index from scratch we handle the transition
using aliases, for example we'd start off with garments-2012-04-01
having the alias garments, then we'd build garments-2012-05-01 and
when we're done we change the aliases around so that garments is now
an alias for garments-2012-05-01. The app always queries the index
using the garments aliases and so it gets a smooth transition from 1
index to another.

We use to do this by doing a delete on garments (which thus deleting
garments-2012-04-01) and then adding the garments alias to
garments-2012-05-01. While this obviously created a window in which
queries would fail, what I had forgotten about was that since some
apis automatically create the index, there was a race condition where
a garments index would be created automatically and my adding the new
alias would fail, so the app is left using the empty auto created
index.

I was considering shuffling the indexes around by removing and adding
the index in one call to the aliases api, something like

curl -XPOST 'http://localhost:9200/_aliases' -d '
{
"actions" : [
{ "remove" : { "index" : "garments-2012-04-01", "alias" :
"garments" } },
{ "add" : { "index" : "garments-2012-05-01", "alias" :
"garments" } }
]
}'

Does this avoid the pitfall I encountered previously or am I better
off just turning off action.auto_create_index ?

Thanks for any advice,

Fred


(Frederick Cheung) #4

On May 9, 9:01 pm, Shay Banon kim...@gmail.com wrote:

Yes, thats the idea of the aliases API, the "operations" that you specify
there are atomic (all or nothing). Note, teh remove simply states that the
alias will be removed, not that the index will be removed/deleted.

Excellent, thanks for the info.

Fred

Also, you can set action.auto_create_index to false in the config file
which will disable automatically creating an index if it does not exists
with the index API.

On Wed, May 9, 2012 at 5:10 PM, Frederick Cheung <frederick.che...@gmail.com

wrote:
Hi,

When we need to rebuild an index from scratch we handle the transition
using aliases, for example we'd start off with garments-2012-04-01
having the alias garments, then we'd build garments-2012-05-01 and
when we're done we change the aliases around so that garments is now
an alias for garments-2012-05-01. The app always queries the index
using the garments aliases and so it gets a smooth transition from 1
index to another.

We use to do this by doing a delete on garments (which thus deleting
garments-2012-04-01) and then adding the garments alias to
garments-2012-05-01. While this obviously created a window in which
queries would fail, what I had forgotten about was that since some
apis automatically create the index, there was a race condition where
a garments index would be created automatically and my adding the new
alias would fail, so the app is left using the empty auto created
index.

I was considering shuffling the indexes around by removing and adding
the index in one call to the aliases api, something like

curl -XPOST 'http://localhost:9200/_aliases'-d '
{
"actions" : [
{ "remove" : { "index" : "garments-2012-04-01", "alias" :
"garments" } },
{ "add" : { "index" : "garments-2012-05-01", "alias" :
"garments" } }
]
}'

Does this avoid the pitfall I encountered previously or am I better
off just turning off action.auto_create_index ?

Thanks for any advice,

Fred


(system) #5