Best practice to backup index daily?


(IronMike) #1

I am able to take a snapshot of the index and back it up to AWS S3. What is
the best way to automate this approach and have it done daily say every
days at 12 midnight?
I am aware that I can probably do it with crontab, but curious if other are
doing it differently?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8620f7d9-b827-470d-8928-75c308e722cc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Ivan Brusic) #2

The Elasticsearch curator now supports snapshots:


You would still need to use cron to schedule tasks, but it would be a
curator task instead of a direct curl request.

Cheers,

Ivan

On Mon, Jul 7, 2014 at 1:12 PM, sabdalla80 sabdalla80@gmail.com wrote:

I am able to take a snapshot of the index and back it up to AWS S3. What
is the best way to automate this approach and have it done daily say every
days at 12 midnight?
I am aware that I can probably do it with crontab, but curious if other
are doing it differently?

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/8620f7d9-b827-470d-8928-75c308e722cc%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/8620f7d9-b827-470d-8928-75c308e722cc%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQB%3D%2BXEGQB3GybgiE3sniiFavc_NMyTK6LgjNnbus4J-jA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(IronMike) #3

This looks great. However I am not sure if I am missing anything, When I
take a snapshot with curl, it works fine by taking the snapshot:

curl -XPUT
http:local_host:9200/_snapshot/es_repository/snapshot_1?wait_for_completion=true
However with curator, it completes but no snapshots are actually taken when
I check, Any ideas what I am missing?
curator snapshot --most-recent 3 --repository es_repository

2014-07-08T17:51:20.143 INFO main:644 Job starting
...

2014-07-08T17:51:20.144 INFO main:654 Default
timeout of 30 seconds is too low for command SNAPSHOT. Overriding to 21,600
seconds (6 hours).

2014-07-08T17:51:20.144 INFO _new_conn:188 Starting new
HTTP connection (1): localhost

2014-07-08T17:51:20.153 INFO log_request_success:57 GET http://localhost:9200/
[status:200 request:0.009s]

2014-07-08T17:51:20.154 INFO command_loop:538 Beginning
SNAPSHOT operations...

2014-07-08T17:51:20.158 INFO log_request_success:57 GET http://localhost:9200/logstash-*/_settings?expand_wildcards=closed
[status:200 request:0.003s]

2014-07-08T17:51:20.158 INFO snap_latest_indices:508 Snapshot
'latest' 3 indices operations completed.

2014-07-08T17:51:20.158 INFO main:671 Done in 0:00:
00.033295.

On Monday, July 7, 2014 6:06:06 PM UTC-4, Ivan Brusic wrote:

The Elasticsearch curator now supports snapshots:

https://github.com/elasticsearch/curator

http://www.elasticsearch.org/blog/elasticsearch-curator-version-1-1-0-released/

You would still need to use cron to schedule tasks, but it would be a
curator task instead of a direct curl request.

Cheers,

Ivan

On Mon, Jul 7, 2014 at 1:12 PM, sabdalla80 <sabda...@gmail.com
<javascript:>> wrote:

I am able to take a snapshot of the index and back it up to AWS S3. What
is the best way to automate this approach and have it done daily say every
days at 12 midnight?
I am aware that I can probably do it with crontab, but curious if other
are doing it differently?

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/8620f7d9-b827-470d-8928-75c308e722cc%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/8620f7d9-b827-470d-8928-75c308e722cc%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/7bc870f5-470a-432a-9217-e30c4002543f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(system) #4