Indexing to Alias

Hi again,

I'm currently on Elasticsearch 0.20.2 was curious to find ES not indexing
any new documents when an alias points to 2 indices.

Example.

index: tweets1 -> alias: tweets
index: tweets2 -> alias: tweets

In this case, I'm doing a PUT on /tweets/ and nothing gets index.

If I tried removing one the alias, which means /tweets/ would point only to
/tweets1/ then the indexing resumes.

Not sure if this is intended or not. One reason I doing the aliasing is
that I could make a "copy" the index for backup and experimentation, since
ES does not support copying or remaining of indices atm i think.

Cheers,
Joshua

--

Hi,

As far as I know you can not index to alias pointing to more indices. That
is expecteed.

If you want to replicate index content to another one you need to do it on
the client side (but should be easy, look at "scroll" & "scan")

Regards,
Lukáš
Dne 19.1.2013 18:20 joshua@zopim.com napsal(a):

Hi again,

I'm currently on Elasticsearch 0.20.2 was curious to find ES not indexing
any new documents when an alias points to 2 indices.

Example.

index: tweets1 -> alias: tweets
index: tweets2 -> alias: tweets

In this case, I'm doing a PUT on /tweets/ and nothing gets index.

If I tried removing one the alias, which means /tweets/ would point only
to /tweets1/ then the indexing resumes.

Not sure if this is intended or not. One reason I doing the aliasing is
that I could make a "copy" the index for backup and experimentation, since
ES does not support copying or remaining of indices atm i think.

Cheers,
Joshua

--

--

Hi!

Thanks for the clarification.

What was unexpected as that an alias pointing to a single index allows
indexing but an alias pointing to multiple indices do not allow. From
a consistency point of view, that was confusing - I would rather have
indexing be disallowed on any alias (even if its just a single index) to
prevent the behavior of indexing on alias which may fail silently.

I have used "scroll and scan", but it does not work for my use cases -
while source data are indexed, they are not stored, so the only way to
recreate the index is to re-index it from the my datasource. It's not a
really an issue, but it would be great that ES have a index
renaming/replication function. I'm sure the speed ES does replication (like
turning on replicas within indices) is much faster than re-indexing data
from the client side.

Cheers,
Joshua

On Sun, Jan 20, 2013 at 2:51 AM, Lukáš Vlček lukas.vlcek@gmail.com wrote:

Hi,

As far as I know you can not index to alias pointing to more indices. That
is expecteed.

If you want to replicate index content to another one you need to do it on
the client side (but should be easy, look at "scroll" & "scan")

Regards,
Lukáš
Dne 19.1.2013 18:20 joshua@zopim.com napsal(a):

Hi again,

I'm currently on Elasticsearch 0.20.2 was curious to find ES not indexing
any new documents when an alias points to 2 indices.

Example.

index: tweets1 -> alias: tweets
index: tweets2 -> alias: tweets

In this case, I'm doing a PUT on /tweets/ and nothing gets index.

If I tried removing one the alias, which means /tweets/ would point only
to /tweets1/ then the indexing resumes.

Not sure if this is intended or not. One reason I doing the aliasing is
that I could make a "copy" the index for backup and experimentation, since
ES does not support copying or remaining of indices atm i think.

Cheers,
Joshua

--

--

--

Hi,

On Tue, Jan 22, 2013 at 4:38 AM, Joshua Koo joshua@zopim.com wrote:

Hi!

Thanks for the clarification.

What was unexpected as that an alias pointing to a single index allows
indexing but an alias pointing to multiple indices do not allow. From
a consistency point of view, that was confusing - I would rather have
indexing be disallowed on any alias (even if its just a single index) to
prevent the behavior of indexing on alias which may fail silently.

I do not think there is a reason why to disable indexing into index with a
single alias. In fast this proved to be useful feature. For example have a
look at rolling index plugin [1,2].

I have used "scroll and scan", but it does not work for my use cases -
while source data are indexed, they are not stored, so the only way to
recreate the index is to re-index it from the my datasource. It's not a
really an issue, but it would be great that ES have a index
renaming/replication function. I'm sure the speed ES does replication (like
turning on replicas within indices) is much faster than re-indexing data
from the client side.

It might be a nice feature but I do not think it is available out of the
box now. In my opinion it might make things a little bit more complex. What
happens when you index [index in terms of mirroring] into two (or more)
indices and the index operation fails with one of the indices? You need to
reflect that in response or try to fix automatically?

I think it is much easier for you to do that on your side.

[1] https://github.com/karussell/elasticsearch-rollindex
[2]
https://groups.google.com/forum/#!topic/elasticsearch/Q_oufvFNEEw/discussion

My 2 cents,
Regards
Lukas

--

Hi Lukas,

That's an interesting plugin, and certainly I can understand the current
behavior now. As for if multi-indices alias indexing would be supported,
I'd think it would have a similar behavior to if there's 2 replicas for a
index and 1 replica failed during an index operation. Of course, just my 2
cents too :wink:

Cheers,
Joshua
p.s. thanks for the work on bigdesk!

On Tue, Jan 22, 2013 at 5:50 PM, Lukáš Vlček lukas.vlcek@gmail.com wrote:

Hi,

On Tue, Jan 22, 2013 at 4:38 AM, Joshua Koo joshua@zopim.com wrote:

Hi!

Thanks for the clarification.

What was unexpected as that an alias pointing to a single index allows
indexing but an alias pointing to multiple indices do not allow. From
a consistency point of view, that was confusing - I would rather have
indexing be disallowed on any alias (even if its just a single index) to
prevent the behavior of indexing on alias which may fail silently.

I do not think there is a reason why to disable indexing into index with a
single alias. In fast this proved to be useful feature. For example have a
look at rolling index plugin [1,2].

I have used "scroll and scan", but it does not work for my use cases -
while source data are indexed, they are not stored, so the only way to
recreate the index is to re-index it from the my datasource. It's not a
really an issue, but it would be great that ES have a index
renaming/replication function. I'm sure the speed ES does replication (like
turning on replicas within indices) is much faster than re-indexing data
from the client side.

It might be a nice feature but I do not think it is available out of the
box now. In my opinion it might make things a little bit more complex. What
happens when you index [index in terms of mirroring] into two (or more)
indices and the index operation fails with one of the indices? You need to
reflect that in response or try to fix automatically?

I think it is much easier for you to do that on your side.

[1] https://github.com/karussell/elasticsearch-rollindex
[2]
https://groups.google.com/forum/#!topic/elasticsearch/Q_oufvFNEEw/discussion

My 2 cents,
Regards
Lukas

--

--