When we need to rebuild an index from scratch we handle the transition
using aliases, for example we'd start off with garments-2012-04-01
having the alias garments, then we'd build garments-2012-05-01 and
when we're done we change the aliases around so that garments is now
an alias for garments-2012-05-01. The app always queries the index
using the garments aliases and so it gets a smooth transition from 1
index to another.
We use to do this by doing a delete on garments (which thus deleting
garments-2012-04-01) and then adding the garments alias to
garments-2012-05-01. While this obviously created a window in which
queries would fail, what I had forgotten about was that since some
apis automatically create the index, there was a race condition where
a garments index would be created automatically and my adding the new
alias would fail, so the app is left using the empty auto created
index.
I was considering shuffling the indexes around by removing and adding
the index in one call to the aliases api, something like
When we need to rebuild an index from scratch we handle the transition
using aliases, for example we'd start off with garments-2012-04-01
having the alias garments, then we'd build garments-2012-05-01 and
when we're done we change the aliases around so that garments is now
an alias for garments-2012-05-01. The app always queries the index
using the garments aliases and so it gets a smooth transition from 1
index to another.
We use to do this by doing a delete on garments (which thus deleting
garments-2012-04-01) and then adding the garments alias to
garments-2012-05-01. While this obviously created a window in which
queries would fail, what I had forgotten about was that since some
apis automatically create the index, there was a race condition where
a garments index would be created automatically and my adding the new
alias would fail, so the app is left using the empty auto created
index.
I was considering shuffling the indexes around by removing and adding
the index in one call to the aliases api, something like
Yes, thats the idea of the aliases API, the "operations" that you specify
there are atomic (all or nothing). Note, teh remove simply states that the
alias will be removed, not that the index will be removed/deleted.
Also, you can set action.auto_create_index to false in the config file
which will disable automatically creating an index if it does not exists
with the index API.
When we need to rebuild an index from scratch we handle the transition
using aliases, for example we'd start off with garments-2012-04-01
having the alias garments, then we'd build garments-2012-05-01 and
when we're done we change the aliases around so that garments is now
an alias for garments-2012-05-01. The app always queries the index
using the garments aliases and so it gets a smooth transition from 1
index to another.
We use to do this by doing a delete on garments (which thus deleting
garments-2012-04-01) and then adding the garments alias to
garments-2012-05-01. While this obviously created a window in which
queries would fail, what I had forgotten about was that since some
apis automatically create the index, there was a race condition where
a garments index would be created automatically and my adding the new
alias would fail, so the app is left using the empty auto created
index.
I was considering shuffling the indexes around by removing and adding
the index in one call to the aliases api, something like
Yes, thats the idea of the aliases API, the "operations" that you specify
there are atomic (all or nothing). Note, teh remove simply states that the
alias will be removed, not that the index will be removed/deleted.
Excellent, thanks for the info.
Fred
Also, you can set action.auto_create_index to false in the config file
which will disable automatically creating an index if it does not exists
with the index API.
When we need to rebuild an index from scratch we handle the transition
using aliases, for example we'd start off with garments-2012-04-01
having the alias garments, then we'd build garments-2012-05-01 and
when we're done we change the aliases around so that garments is now
an alias for garments-2012-05-01. The app always queries the index
using the garments aliases and so it gets a smooth transition from 1
index to another.
We use to do this by doing a delete on garments (which thus deleting
garments-2012-04-01) and then adding the garments alias to
garments-2012-05-01. While this obviously created a window in which
queries would fail, what I had forgotten about was that since some
apis automatically create the index, there was a race condition where
a garments index would be created automatically and my adding the new
alias would fail, so the app is left using the empty auto created
index.
I was considering shuffling the indexes around by removing and adding
the index in one call to the aliases api, something like
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.