Real time Migration of Tenant/Customer from One Cluster to Another Cluster

Elastic Stack Elasticsearch
1

Hi,

We are having multiple tenants in one cluster. And since the volume of one
tenant has increased, we are planning to move that Tenant into different
Cluster.
The only challenge is - it needs to be done in Real time without Customer
Impact.

Does anyone know if Elasticsearch has some functionality to do this kind of
migration of Tenant data (from one cluster to another) in Real time?
Or any other Ideas?

Thanks for support.

Thanks
Pranav.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

You could use logstash's elasticsearch in/outputs, or this
GitHub - elastic/stream2es: Stream data into ES (Wikipedia, Twitter, stdin, or other ESes).

V1.0 of elasticsearch will contain snapshot/backup functionality that will
eventually allow this.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 21 November 2013 09:16, pranav amin parulpatel25@gmail.com wrote:

Hi,

We are having multiple tenants in one cluster. And since the volume of one
tenant has increased, we are planning to move that Tenant into different
Cluster.
The only challenge is - it needs to be done in Real time without Customer
Impact.

Does anyone know if Elasticsearch has some functionality to do this kind
of migration of Tenant data (from one cluster to another) in Real time?
Or any other Ideas?

Thanks for support.

Thanks
Pranav.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

Hi Mark,

Thanks for responding.

Just want to make sure can i use them in real time so that the customer
doesn't any impact as the data is moved to other cluster?
So basically A customer has 10 Million records, and it is a LIVE
environment - how do i start the migration?
I mean do i just start the [snapshot/restore or logstash or github link]
and it will make sure whatever data is there in Old Cluster along with the
Ongoing changes move to New cluster?

Thanks
Pranav.

On Wed, Nov 20, 2013 at 5:19 PM, Mark Walkom markw@campaignmonitor.comwrote:

You could use logstash's elasticsearch in/outputs, or this
GitHub - elastic/stream2es: Stream data into ES (Wikipedia, Twitter, stdin, or other ESes).

V1.0 of elasticsearch will contain snapshot/backup functionality that will
eventually allow this.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 21 November 2013 09:16, pranav amin parulpatel25@gmail.com wrote:

Hi,

We are having multiple tenants in one cluster. And since the volume of
one tenant has increased, we are planning to move that Tenant into
different Cluster.
The only challenge is - it needs to be done in Real time without
Customer Impact.

Does anyone know if Elasticsearch has some functionality to do this kind
of migration of Tenant data (from one cluster to another) in Real time?
Or any other Ideas?

Thanks for support.

Thanks
Pranav.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/yR_od8zPL-I/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

4

I can't comment on that as I haven't yet had the chance to use stream2es
(but will soon). You might want to test the process first to make sure it
does what you want.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 22 November 2013 07:22, parul patel parulpatel25@gmail.com wrote:

Hi Mark,

Thanks for responding.

Just want to make sure can i use them in real time so that the customer
doesn't any impact as the data is moved to other cluster?
So basically A customer has 10 Million records, and it is a LIVE
environment - how do i start the migration?
I mean do i just start the [snapshot/restore or logstash or github link]
and it will make sure whatever data is there in Old Cluster along with the
Ongoing changes move to New cluster?

Thanks
Pranav.

On Wed, Nov 20, 2013 at 5:19 PM, Mark Walkom markw@campaignmonitor.comwrote:

You could use logstash's elasticsearch in/outputs, or this
GitHub - elastic/stream2es: Stream data into ES (Wikipedia, Twitter, stdin, or other ESes).

V1.0 of elasticsearch will contain snapshot/backup functionality that
will eventually allow this.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 21 November 2013 09:16, pranav amin parulpatel25@gmail.com wrote:

Hi,

We are having multiple tenants in one cluster. And since the volume of
one tenant has increased, we are planning to move that Tenant into
different Cluster.
The only challenge is - it needs to be done in Real time without
Customer Impact.

Does anyone know if Elasticsearch has some functionality to do this kind
of migration of Tenant data (from one cluster to another) in Real time?
Or any other Ideas?

Thanks for support.

Thanks
Pranav.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/yR_od8zPL-I/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.