Transitioning percolation from 0.9 to 1.0

Elastic Stack Elasticsearch
1

Is there a formal plan for how to transition between the old percolator
mechanism and the new one?

More specifically, does the old percolator mechanism still work in 1.0, in
addition to the new distributed one? Or is just the latter?

I ask, because I'm wondering how we're going to upgrade our two (redundant)
clusters to 1.0 from .9 without breaking everything that uses percolate.
If I could count on the _percolator index still existing and working, we
could upgrade with brief outages, and then slowly transition to the new
distributed mechanism; but as far as I can tell, with only the distributed
mechanism up, I'd need to bring the old cluster down, bring the new cluster
up, and then spend the time transitioning the _percolator index to it's
.percolator counter part.

Anyone have any experience with such a transition before?

-Adam

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/41ad7018-d48c-438b-abbc-690b499bfadf%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

Adam,

You'll likely need to bring it down and transition all the percolator
queries as well as your code before you can bring in back up in 1.0. The
best way to do this is to start with the transition using the latest 1.0 RC
releases. Once you adjust everything to the new .percolator index and API,
you should be good to go for 1.0.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/90d4d46b-610c-484a-b0af-31790d721f17%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

And I forgot to mention, index names cannot start with "_" anymore so
definitely you will not be able to transition the "_percolator" index name.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/3dcf42fa-b9d5-4d69-8bc8-a7d617a05a0b%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

4

Hi Adam,
the important part is that the queries can now be indexed under any index
(thus they can scale out!), and they get stored under the .percolator type.

Therefore instead of having your _percolator index you'll have to pick a
name for your index, and migrate the queries, which will go to the
.percolator type this time. That said features have been added, like
percolating an existing document, highlighting etc. The only feature that
has been removed is the percolation while indexing, which can be easily
replaced with two calls, first one to index the doc and second one to
percolate the existing doc (works in real-time as it uses the get api
internally).

Hope this helps
Luca

On Wednesday, January 29, 2014 6:12:03 PM UTC+1, Binh Ly wrote:

And I forgot to mention, index names cannot start with "_" anymore so
definitely you will not be able to transition the "_percolator" index name.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/951125b1-6c15-4565-a069-3a3c8378a0d9%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

5

That's how i understood it, too. But I was hoping to be able to keep both
the _percolator and new .peroclator up, index to both after back-filling
the new .percolator portion of the index, then switch over my _percolate
calls after both indexes were in sync.

The way things currently stand, unless I keep both my current .9 cluster
and the new 1.0 cluster up simultaneously, I can't do that.

Oh well...

Thanks for the help, friends.

-Adam

On Wednesday, January 29, 2014 1:13:27 PM UTC-5, Luca Cavanna wrote:

Hi Adam,
the important part is that the queries can now be indexed under any index
(thus they can scale out!), and they get stored under the .percolator type.

Therefore instead of having your _percolator index you'll have to pick a
name for your index, and migrate the queries, which will go to the
.percolator type this time. That said features have been added, like
percolating an existing document, highlighting etc. The only feature that
has been removed is the percolation while indexing, which can be easily
replaced with two calls, first one to index the doc and second one to
percolate the existing doc (works in real-time as it uses the get api
internally).

Hope this helps
Luca

On Wednesday, January 29, 2014 6:12:03 PM UTC+1, Binh Ly wrote:

And I forgot to mention, index names cannot start with "_" anymore so
definitely you will not be able to transition the "_percolator" index name.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/878844c8-cd61-49b5-a8f8-dd4d7050ad4b%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

6

Good news, you can temporarily keep the _percolator index, no need to have
both clusters running for the migration.
In 1.0, the existing _percolator index will be picked up and will be
available, but only as a normal index (no percolations against those
queries). That means you can use a scan search, reindex your queries into
the new index/.percolator, and finally delete the _percolator index.

On Wed, Jan 29, 2014 at 7:29 PM, Adam Georgiou apg552@gmail.com wrote:

That's how i understood it, too. But I was hoping to be able to keep both
the _percolator and new .peroclator up, index to both after back-filling
the new .percolator portion of the index, then switch over my _percolate
calls after both indexes were in sync.

The way things currently stand, unless I keep both my current .9 cluster
and the new 1.0 cluster up simultaneously, I can't do that.

Oh well...

Thanks for the help, friends.

-Adam

On Wednesday, January 29, 2014 1:13:27 PM UTC-5, Luca Cavanna wrote:

Hi Adam,
the important part is that the queries can now be indexed under any index
(thus they can scale out!), and they get stored under the .percolator type.

Therefore instead of having your _percolator index you'll have to pick a
name for your index, and migrate the queries, which will go to the
.percolator type this time. That said features have been added, like
percolating an existing document, highlighting etc. The only feature that
has been removed is the percolation while indexing, which can be easily
replaced with two calls, first one to index the doc and second one to
percolate the existing doc (works in real-time as it uses the get api
internally).

Hope this helps
Luca

On Wednesday, January 29, 2014 6:12:03 PM UTC+1, Binh Ly wrote:

And I forgot to mention, index names cannot start with "_" anymore so
definitely you will not be able to transition the "_percolator" index name.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/RT0SCzkFu0I/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/878844c8-cd61-49b5-a8f8-dd4d7050ad4b%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CADdZ9MXfM2jotAO6nLMFu-y7MCfoNv9nQyddagEKDXZkpSBhPg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.