Backup procedure for ES nodes

vijayprabhakar · July 2, 2013, 1:51pm

Hi Team

I need some backup policies for my environment.
I am using 5 clusters with 3 nodes which is spread on 3 machines.
I need to take backup of all indexes in a cluster.

So what i thinking is shall i copy and archive the "data" directory alone..is this correct ..?

Also its enough to take the backup from any one node for a cluster, Whether using that i can able to recover the cluster ..?

radu_gheorghe · July 10, 2013, 7:47am

Hello,

Before backing up your data directory, you need to disable flush, so you
don't catch it during a flush and get an inconsistent state:

curl -XPUT "$ADDRESS/$INDEX_NAME/_settings" -d '{
"index": {
"translog.disable_flush": true
}}'

After your backup is done, you should enable flush back again.

You only need to do that one one node if that node contains a complete set
of shards. For example, if you have 3 nodes, and all your indices have 2
replicas, then it should work. If you have 0 or one replicas, it's best to
backup all data directories.

Here are two scripts I've tested for backing up and restoring "old" indices
containing logs, in case it gives any insight:

gist.github.com

https://gist.github.com/radu-gheorghe/3180985

log_backup.bash

#!/usr/bin/env bash

###############FUNCTIONS############

function prepare {
    #optimize the index
    echo -n "Optimizing index $INDEX_NAME..."
    curl -XPOST "$ADDRESS/$INDEX_NAME/_optimize" 2>/dev/null| grep 'failed":0' >/dev/null
    if [ $? -eq 0 ]; then
        echo "done"

This file has been truncated. show original

log_restore.bash

#!/usr/bin/env bash

###############FUNCTIONS############

function get_arguments {
    for ARGUMENT in "$@"; do
        case "$ARGUMENT" in
            -h|--help)
                echo "This will restpre indices from backup tp Elasticsearch."
                echo

This file has been truncated. show original

I was also optimizing them, because they wouldn't change from that point
on. I don't know it it makes sense for you or not.

Best regards,
Radu

On Tue, Jul 2, 2013 at 4:51 PM, vijayprabhakar
vijayprabhakar.90@gmail.comwrote:

Hi Team

I need some backup policies for my environment.
I am using 5 clusters with 3 nodes which is spread on 3 machines.
I need to take backup of all indexes in a cluster.

So what i thinking is shall i copy and archive the "data" directory
alone..is this correct ..?

Also its enough to take the backup from any one node for a cluster, Whether
using that i can able to recover the cluster ..?

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/Backup-procedure-for-ES-nodes-tp4037374.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
http://sematext.com/ -- Elasticsearch -- Solr -- Lucene

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Topic		Replies	Views
Backup Policies for ES Elasticsearch	4	420	July 6, 2017
Elasticsearch - backup a cluster Elasticsearch	3	365	January 17, 2019
ElasticSearch 1.0 Manual Backup Elasticsearch	4	568	July 6, 2017
How to (offsite) backup and recover snapshots Elasticsearch	6	425	September 12, 2022
Best way to backup Elasticsearch	2	837	July 5, 2017

Backup procedure for ES nodes

Related topics