How to store a backup (snapshot) of my indexes to local storage using python in elasticsearch and then also be able to restore them?

Elastic Stack Elasticsearch
1

Hello,

I know that I need to create a snapshot (?) for doing so, but can someone please tell me how?

Say I have indexes: indexA and indexB

How do I go about storing a snapshot (containing these indexes) to local storage using elasticsearch on python?

It would help if you could also tell me how to restore them.

2

first of all, you should indicate your ES version.
anyway, you can follow the guide here

basically, you have to create an Elasticsearch connection object (i.e. es) and then you can use the standard function es.snapshot.create and es.snapshot.restore

for instance:

es.snapshot.create(repository=repository,snapshot=sname,wait_for_completion="true",body=settingsSnap)

where:

repository = "myfsrepo"
today = datetime.datetime.today()
timestamp = today.strftime("%Y.%m.%d")
snapshot = "snaptest_" + timestamp
settingsSnap = {
"indices": indexA,
"ignore_unavailable": "true",
"include_global_state": "true"
}

to restore it, you can work in a similar way.
Remember that before to work with Python, you have to configure the path.repo key value in your elasticsearch.yml and create your repository (maybe using Kibana if you prefer).

3

This is my version information:

{'name': '', 'cluster_name': '', 'cluster_uuid': '', 'version': {'number': '6.7.2', 'bui
ld_flavor': 'oss', 'build_type': 'tar', 'build_hash': '', 'build_date': :
  , 'build_snapshot': False, 'lucene_version': '7.7.0', 'minimum_wire_compatibilit
y_version': '5.6.0', 'minimum_index_compatibility_version': '5.0.0'}, 'tagline': 'You Know, for Search'}

Will the above work for storing it on local storage somewhere? For eg. if I wanted to store the ES indexes into /home/usr1/indexes/ ?

4

I tried this:

snapshot_body = {
"type": "url",
"settings": {
        "url":  "http://download.elasticsearch.org/definitiveguide/sigterms_demo/"
    }
}
body = {"snapshot": snapshot_body}
es.snapshot.create_repository(repository='test', body=snapshot_body)
index_body = {
  "indices":"index1,index2" 
}
es.snapshot.create(repository='test', snapshot='my_snapshot', body=index_body)

getting error:

elasticsearch.exceptions.TransportError: TransportError(500, 'repository_exception', "[test] file url [http://download.elasticsearch.org/definitiveguide/sigterms_demo/] doesn't match any of the locations specified by path.repo or repositories.url.allowed_urls")

Am I creating the snapshot correctly?

I want to store the data from index1,index2 into a path /home/usr1/indexes/

I know that I can manually scroll through the index data and then save it but I also read somewhere that it is not recommended. Can someone please give me a small working example which they tested on their side?

5

Can someone please answer this query? I don't think the response appropriately answers the question.

6

Snapshots in Elasticsearch require shared storage which all the nodes in the cluster have access to on the same path. Local storage can only be used if you only have a single nodes/host.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.