Details on snapshot and restore in ES 1.0


(Hari Prasad) #1

Hi
I am try to know about the new feature snapshot and restore in the
Elasticsearch 1.0. Can anyone provide the details or provide a link that
gives a detailed explanation on this, other that the official blog. I was
not about find enough info on it.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/07116855-538f-41b3-9ba9-2eca919e3385%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Clinton Gormley) #2

Here is a link to the docs:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-snapshots.html

Without knowing what you are missing it is difficult to provide more
information

Clint

On 5 March 2014 11:01, Hari Prasad iamhari1234@gmail.com wrote:

Hi
I am try to know about the new feature snapshot and restore in the
Elasticsearch 1.0. Can anyone provide the details or provide a link that
gives a detailed explanation on this, other that the official blog. I was
not about find enough info on it.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/07116855-538f-41b3-9ba9-2eca919e3385%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/07116855-538f-41b3-9ba9-2eca919e3385%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPt3XKTawR2Rty7BrAEtXjxXAAxEx4WB2XXBR7AGeFcpu1S%2Bqg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Hari Prasad) #3

Thank you Clint for quick response.
I would like to know about the read-only url repository. Whats is its use
and possible use cases?

On Wednesday, 5 March 2014 15:35:53 UTC+5:30, Clinton Gormley wrote:

Here is a link to the docs:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-snapshots.html

Without knowing what you are missing it is difficult to provide more
information

Clint

On 5 March 2014 11:01, Hari Prasad <iamha...@gmail.com <javascript:>>wrote:

Hi
I am try to know about the new feature snapshot and restore in the
Elasticsearch 1.0. Can anyone provide the details or provide a link that
gives a detailed explanation on this, other that the official blog. I was
not about find enough info on it.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/07116855-538f-41b3-9ba9-2eca919e3385%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/07116855-538f-41b3-9ba9-2eca919e3385%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/25ed4855-066e-4b8b-8179-0412d625b1f4%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Clinton Gormley) #4

For instance, you could snapshot an index, put it somewhere on the web, and
allow people to restore from that URL into their own cluster.

clint

On 5 March 2014 11:31, Hari Prasad iamhari1234@gmail.com wrote:

Thank you Clint for quick response.
I would like to know about the read-only url repository. Whats is its use
and possible use cases?

On Wednesday, 5 March 2014 15:35:53 UTC+5:30, Clinton Gormley wrote:

Here is a link to the docs: http://www.elasticsearch.org/guide/en/
elasticsearch/reference/current/modules-snapshots.html

Without knowing what you are missing it is difficult to provide more
information

Clint

On 5 March 2014 11:01, Hari Prasad iamha...@gmail.com wrote:

Hi
I am try to know about the new feature snapshot and restore in the
Elasticsearch 1.0. Can anyone provide the details or provide a link that
gives a detailed explanation on this, other that the official blog. I was
not about find enough info on it.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/07116855-538f-41b3-9ba9-2eca919e3385%
40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/07116855-538f-41b3-9ba9-2eca919e3385%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/25ed4855-066e-4b8b-8179-0412d625b1f4%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/25ed4855-066e-4b8b-8179-0412d625b1f4%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPt3XKRfhTb6XdN9GrZEaEzMrs97%2BMNiswPAszCBcbGJ2fHqYw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Hari Prasad) #5

So can a snapshot taken for one cluster be restored for another cluster,
may be with different name?

On Wednesday, 5 March 2014 17:04:29 UTC+5:30, Clinton Gormley wrote:

For instance, you could snapshot an index, put it somewhere on the web,
and allow people to restore from that URL into their own cluster.

clint

On 5 March 2014 11:31, Hari Prasad <iamha...@gmail.com <javascript:>>wrote:

Thank you Clint for quick response.
I would like to know about the read-only url repository. Whats is its use
and possible use cases?

On Wednesday, 5 March 2014 15:35:53 UTC+5:30, Clinton Gormley wrote:

Here is a link to the docs: http://www.elasticsearch.org/guide/en/
elasticsearch/reference/current/modules-snapshots.html

Without knowing what you are missing it is difficult to provide more
information

Clint

On 5 March 2014 11:01, Hari Prasad iamha...@gmail.com wrote:

Hi
I am try to know about the new feature snapshot and restore in the
Elasticsearch 1.0. Can anyone provide the details or provide a link that
gives a detailed explanation on this, other that the official blog. I was
not about find enough info on it.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/07116855-538f-41b3-9ba9-2eca919e3385%
40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/07116855-538f-41b3-9ba9-2eca919e3385%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/25ed4855-066e-4b8b-8179-0412d625b1f4%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/25ed4855-066e-4b8b-8179-0412d625b1f4%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/bb7c28e9-f6eb-42af-b0ea-be8f8ef6cc68%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Clinton Gormley) #6

On 5 March 2014 12:38, Hari Prasad iamhari1234@gmail.com wrote:

So can a snapshot taken for one cluster be restored for another cluster,
may be with different name?

yes

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPt3XKSb_6BshJs5rmV8efwg0iXuAQbnWOr9uR2cZfKq2%2B%3Df5g%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Amit Soni) #7

In the article above, it says "At the time of the initial release only shared
file system
repository is supported."

Can you pl clarify on this?

-Amit.

On Wed, Mar 5, 2014 at 3:38 AM, Clinton Gormley clint@traveljury.comwrote:

On 5 March 2014 12:38, Hari Prasad iamhari1234@gmail.com wrote:

So can a snapshot taken for one cluster be restored for another cluster,
may be with different name?

yes

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAPt3XKSb_6BshJs5rmV8efwg0iXuAQbnWOr9uR2cZfKq2%2B%3Df5g%40mail.gmail.comhttps://groups.google.com/d/msgid/elasticsearch/CAPt3XKSb_6BshJs5rmV8efwg0iXuAQbnWOr9uR2cZfKq2%2B%3Df5g%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAAOGaQLUiaQekq5y_jtb%3Dr2qO9OYFHFw%3DAxamc--mDh%3Djf2Rhg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(David Pilato) #8

It means that you can Backup on file system.
Note that it's now possible to backup to S3 and HDFS. Azure storage coming...

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 6 mars 2014 à 03:27, Amit Soni amitsoni29@gmail.com a écrit :

In the article above, it says "At the time of the initial release only shared file system repository is supported."

Can you pl clarify on this?

-Amit.

On Wed, Mar 5, 2014 at 3:38 AM, Clinton Gormley clint@traveljury.com wrote:

On 5 March 2014 12:38, Hari Prasad iamhari1234@gmail.com wrote:
So can a snapshot taken for one cluster be restored for another cluster, may be with different name?

yes

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPt3XKSb_6BshJs5rmV8efwg0iXuAQbnWOr9uR2cZfKq2%2B%3Df5g%40mail.gmail.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAAOGaQLUiaQekq5y_jtb%3Dr2qO9OYFHFw%3DAxamc--mDh%3Djf2Rhg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/51072F77-AE89-41F4-9816-EDFD17B760A7%40pilato.fr.
For more options, visit https://groups.google.com/groups/opt_out.


(Amit Soni) #9

Thanks David. Actually "shared file system" got me a little confused with
setting up a file system which would need to be mounted on all
participating nodes and that seemed bizarre. Hence wanted to clarify!

-Amit.

On Wed, Mar 5, 2014 at 10:36 PM, David Pilato david@pilato.fr wrote:

It means that you can Backup on file system.
Note that it's now possible to backup to S3 and HDFS. Azure storage
coming...

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 6 mars 2014 à 03:27, Amit Soni amitsoni29@gmail.com a écrit :

In the article above, it says "At the time of the initial release only shared
file system
repository is supported."

Can you pl clarify on this?

-Amit.

On Wed, Mar 5, 2014 at 3:38 AM, Clinton Gormley clint@traveljury.comwrote:

On 5 March 2014 12:38, Hari Prasad iamhari1234@gmail.com wrote:

So can a snapshot taken for one cluster be restored for another cluster,
may be with different name?

yes

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAPt3XKSb_6BshJs5rmV8efwg0iXuAQbnWOr9uR2cZfKq2%2B%3Df5g%40mail.gmail.comhttps://groups.google.com/d/msgid/elasticsearch/CAPt3XKSb_6BshJs5rmV8efwg0iXuAQbnWOr9uR2cZfKq2%2B%3Df5g%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAAOGaQLUiaQekq5y_jtb%3Dr2qO9OYFHFw%3DAxamc--mDh%3Djf2Rhg%40mail.gmail.comhttps://groups.google.com/d/msgid/elasticsearch/CAAOGaQLUiaQekq5y_jtb%3Dr2qO9OYFHFw%3DAxamc--mDh%3Djf2Rhg%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/51072F77-AE89-41F4-9816-EDFD17B760A7%40pilato.frhttps://groups.google.com/d/msgid/elasticsearch/51072F77-AE89-41F4-9816-EDFD17B760A7%40pilato.fr?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAAOGaQ%2BW3eiFMKJeOVRrFLYSqF_Mem-90TSZnQw2ngkhREXP0A%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(David Pilato) #10

Actually, I think that your backup file system needs to be shared on all nodes because you don't know in advance which nodes will actually perform the snapshot.

(Though I need to double check)

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 6 mars 2014 à 08:28, Amit Soni amitsoni29@gmail.com a écrit :

Thanks David. Actually "shared file system" got me a little confused with setting up a file system which would need to be mounted on all participating nodes and that seemed bizarre. Hence wanted to clarify!

-Amit.

On Wed, Mar 5, 2014 at 10:36 PM, David Pilato david@pilato.fr wrote:
It means that you can Backup on file system.
Note that it's now possible to backup to S3 and HDFS. Azure storage coming...

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 6 mars 2014 à 03:27, Amit Soni amitsoni29@gmail.com a écrit :

In the article above, it says "At the time of the initial release only shared file system repository is supported."

Can you pl clarify on this?

-Amit.

On Wed, Mar 5, 2014 at 3:38 AM, Clinton Gormley clint@traveljury.com wrote:

On 5 March 2014 12:38, Hari Prasad iamhari1234@gmail.com wrote:
So can a snapshot taken for one cluster be restored for another cluster, may be with different name?

yes

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPt3XKSb_6BshJs5rmV8efwg0iXuAQbnWOr9uR2cZfKq2%2B%3Df5g%40mail.gmail.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAAOGaQLUiaQekq5y_jtb%3Dr2qO9OYFHFw%3DAxamc--mDh%3Djf2Rhg%40mail.gmail.com.

For more options, visit https://groups.google.com/groups/opt_out.

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/51072F77-AE89-41F4-9816-EDFD17B760A7%40pilato.fr.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAAOGaQ%2BW3eiFMKJeOVRrFLYSqF_Mem-90TSZnQw2ngkhREXP0A%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CEF176CA-257B-4B17-90AD-15C207002288%40pilato.fr.
For more options, visit https://groups.google.com/groups/opt_out.


(David Pilato) #11

In related issue:

Shared File System Repository

The shared file system repository ("type": "fs") is using shared file system to store snapshot. The path specified in the location parameter should point to the same location in the shared filesystem and be accessible on all data and master nodes.

May be we should add an option to _snapshot endpoint which could be preference=local for example which will execute the snapshot only on the local node which get the request.

Need to talk with Igor about it though to see if it makes sense or is doable.

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 6 mars 2014 à 08:43, David Pilato david@pilato.fr a écrit :

Actually, I think that your backup file system needs to be shared on all nodes because you don't know in advance which nodes will actually perform the snapshot.

(Though I need to double check)

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 6 mars 2014 à 08:28, Amit Soni amitsoni29@gmail.com a écrit :

Thanks David. Actually "shared file system" got me a little confused with setting up a file system which would need to be mounted on all participating nodes and that seemed bizarre. Hence wanted to clarify!

-Amit.

On Wed, Mar 5, 2014 at 10:36 PM, David Pilato david@pilato.fr wrote:
It means that you can Backup on file system.
Note that it's now possible to backup to S3 and HDFS. Azure storage coming...

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 6 mars 2014 à 03:27, Amit Soni amitsoni29@gmail.com a écrit :

In the article above, it says "At the time of the initial release only shared file system repository is supported."

Can you pl clarify on this?

-Amit.

On Wed, Mar 5, 2014 at 3:38 AM, Clinton Gormley clint@traveljury.com wrote:

On 5 March 2014 12:38, Hari Prasad iamhari1234@gmail.com wrote:
So can a snapshot taken for one cluster be restored for another cluster, may be with different name?

yes

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPt3XKSb_6BshJs5rmV8efwg0iXuAQbnWOr9uR2cZfKq2%2B%3Df5g%40mail.gmail.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAAOGaQLUiaQekq5y_jtb%3Dr2qO9OYFHFw%3DAxamc--mDh%3Djf2Rhg%40mail.gmail.com.

For more options, visit https://groups.google.com/groups/opt_out.

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/51072F77-AE89-41F4-9816-EDFD17B760A7%40pilato.fr.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAAOGaQ%2BW3eiFMKJeOVRrFLYSqF_Mem-90TSZnQw2ngkhREXP0A%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CEF176CA-257B-4B17-90AD-15C207002288%40pilato.fr.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/AF824946-2CA8-480F-906D-6134CEC4050D%40pilato.fr.
For more options, visit https://groups.google.com/groups/opt_out.


(Amit Soni) #12

Thanks David, pl do share how your discussion with Igor goes.

I agree that a "preference=local" flag will be useful since having a shared
file system is just additional setup to be done and if there is any way to
avoid it, things would get simpler.

-Amit.

On Wed, Mar 5, 2014 at 11:57 PM, David Pilato david@pilato.fr wrote:

In related issue:

Shared File System Repository

The shared file system repository ("type": "fs") is using shared file
system to store snapshot. The path specified in the location parameter
should point to the same location in the shared filesystem and be
accessible on all data and master nodes.

May be we should add an option to _snapshot endpoint which could be
preference=local for example which will execute the snapshot only on the
local node which get the request.

Need to talk with Igor about it though to see if it makes sense or is
doable.

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 6 mars 2014 à 08:43, David Pilato david@pilato.fr a écrit :

Actually, I think that your backup file system needs to be shared on all
nodes because you don't know in advance which nodes will actually perform
the snapshot.

(Though I need to double check)

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 6 mars 2014 à 08:28, Amit Soni amitsoni29@gmail.com a écrit :

Thanks David. Actually "shared file system" got me a little confused
with setting up a file system which would need to be mounted on all
participating nodes and that seemed bizarre. Hence wanted to clarify!

-Amit.

On Wed, Mar 5, 2014 at 10:36 PM, David Pilato david@pilato.fr wrote:

It means that you can Backup on file system.
Note that it's now possible to backup to S3 and HDFS. Azure storage
coming...

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 6 mars 2014 à 03:27, Amit Soni amitsoni29@gmail.com a écrit :

In the article above, it says "At the time of the initial release only shared
file system
repository is supported."

Can you pl clarify on this?

-Amit.

On Wed, Mar 5, 2014 at 3:38 AM, Clinton Gormley clint@traveljury.comwrote:

On 5 March 2014 12:38, Hari Prasad iamhari1234@gmail.com wrote:

So can a snapshot taken for one cluster be restored for another
cluster, may be with different name?

yes

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAPt3XKSb_6BshJs5rmV8efwg0iXuAQbnWOr9uR2cZfKq2%2B%3Df5g%40mail.gmail.comhttps://groups.google.com/d/msgid/elasticsearch/CAPt3XKSb_6BshJs5rmV8efwg0iXuAQbnWOr9uR2cZfKq2%2B%3Df5g%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAAOGaQLUiaQekq5y_jtb%3Dr2qO9OYFHFw%3DAxamc--mDh%3Djf2Rhg%40mail.gmail.comhttps://groups.google.com/d/msgid/elasticsearch/CAAOGaQLUiaQekq5y_jtb%3Dr2qO9OYFHFw%3DAxamc--mDh%3Djf2Rhg%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/51072F77-AE89-41F4-9816-EDFD17B760A7%40pilato.frhttps://groups.google.com/d/msgid/elasticsearch/51072F77-AE89-41F4-9816-EDFD17B760A7%40pilato.fr?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAAOGaQ%2BW3eiFMKJeOVRrFLYSqF_Mem-90TSZnQw2ngkhREXP0A%40mail.gmail.comhttps://groups.google.com/d/msgid/elasticsearch/CAAOGaQ%2BW3eiFMKJeOVRrFLYSqF_Mem-90TSZnQw2ngkhREXP0A%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CEF176CA-257B-4B17-90AD-15C207002288%40pilato.frhttps://groups.google.com/d/msgid/elasticsearch/CEF176CA-257B-4B17-90AD-15C207002288%40pilato.fr?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/AF824946-2CA8-480F-906D-6134CEC4050D%40pilato.frhttps://groups.google.com/d/msgid/elasticsearch/AF824946-2CA8-480F-906D-6134CEC4050D%40pilato.fr?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAAOGaQL-sQ2Qwr2QWD%2BaUWrtWyBgYz-yU4U4Sbh%3DjEvVXT%3DRVw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(David Pilato) #13

We just talked about it.

Right now, master node needs to save some metadata information in FS and each node which has a primary shard will copy data directly to the filesystem.
That's the reason each master node and data node needs to have access to the file system.

One option could be in the future to copy all data over the wire to master node and the stream data to FS. So only master node will need to have access to the FS repository.
But it's not here yet and we need to think about it as it can come with some implications.

May be a SSH repository could be easier in that case and won't overload master node with tons of network data...

Best

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr

Le 6 mars 2014 à 18:19:43, Amit Soni (amitsoni29@gmail.com) a écrit:

Thanks David, pl do share how your discussion with Igor goes.

I agree that a "preference=local" flag will be useful since having a shared file system is just additional setup to be done and if there is any way to avoid it, things would get simpler.

-Amit.

On Wed, Mar 5, 2014 at 11:57 PM, David Pilato david@pilato.fr wrote:
In related issue:

Shared File System Repository

The shared file system repository ("type": "fs") is using shared file system to store snapshot. The path specified in the location parameter should point to the same location in the shared filesystem and be accessible on all data and master nodes.

May be we should add an option to _snapshot endpoint which could be preference=local for example which will execute the snapshot only on the local node which get the request.

Need to talk with Igor about it though to see if it makes sense or is doable.

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 6 mars 2014 à 08:43, David Pilato david@pilato.fr a écrit :

Actually, I think that your backup file system needs to be shared on all nodes because you don't know in advance which nodes will actually perform the snapshot.

(Though I need to double check)

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 6 mars 2014 à 08:28, Amit Soni amitsoni29@gmail.com a écrit :

Thanks David. Actually "shared file system" got me a little confused with setting up a file system which would need to be mounted on all participating nodes and that seemed bizarre. Hence wanted to clarify!

-Amit.

On Wed, Mar 5, 2014 at 10:36 PM, David Pilato david@pilato.fr wrote:
It means that you can Backup on file system.
Note that it's now possible to backup to S3 and HDFS. Azure storage coming...

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 6 mars 2014 à 03:27, Amit Soni amitsoni29@gmail.com a écrit :

In the article above, it says "At the time of the initial release only shared file system repository is supported."

Can you pl clarify on this?

-Amit.

On Wed, Mar 5, 2014 at 3:38 AM, Clinton Gormley clint@traveljury.com wrote:

On 5 March 2014 12:38, Hari Prasad iamhari1234@gmail.com wrote:
So can a snapshot taken for one cluster be restored for another cluster, may be with different name?

yes

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPt3XKSb_6BshJs5rmV8efwg0iXuAQbnWOr9uR2cZfKq2%2B%3Df5g%40mail.gmail.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAAOGaQLUiaQekq5y_jtb%3Dr2qO9OYFHFw%3DAxamc--mDh%3Djf2Rhg%40mail.gmail.com.

For more options, visit https://groups.google.com/groups/opt_out.

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/51072F77-AE89-41F4-9816-EDFD17B760A7%40pilato.fr.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAAOGaQ%2BW3eiFMKJeOVRrFLYSqF_Mem-90TSZnQw2ngkhREXP0A%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CEF176CA-257B-4B17-90AD-15C207002288%40pilato.fr.
For more options, visit https://groups.google.com/groups/opt_out.

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/AF824946-2CA8-480F-906D-6134CEC4050D%40pilato.fr.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAAOGaQL-sQ2Qwr2QWD%2BaUWrtWyBgYz-yU4U4Sbh%3DjEvVXT%3DRVw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.5318afd2.661e3f1e.7bfa%40MacBook-Air-de-David.local.
For more options, visit https://groups.google.com/groups/opt_out.


(Hari Prasad) #14

Thank you David for response
So untill this new settings, that you mention, will i have to mount the shared location on all the nodes?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/db533b01-075e-4193-99c6-59506d5cd5ed%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(David Pilato) #15

Yes!

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr

Le 6 mars 2014 à 18:32:02, Hari Prasad (iamhari1234@gmail.com) a écrit:

Thank you David for response
So untill this new settings, that you mention, will i have to mount the shared location on all the nodes?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/db533b01-075e-4193-99c6-59506d5cd5ed%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.5318b1a5.51d9c564.7bfa%40MacBook-Air-de-David.local.
For more options, visit https://groups.google.com/groups/opt_out.


(Hari Prasad) #16

Thank you David. That cleared a few things

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/2c824760-0f8b-4f16-ad22-21e6439d8717%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #17