# Migrating elasticsearch data **URL:** \ **Category:** Elasticsearch\ **Created:** [August 27, 2019, 8:40am UTC](https://discuss.elastic.co/t/migrating-elasticsearch-data/196905 "2019-08-27T08:40:25Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![ShahafSW](https://avatars.discourse-cdn.com/v4/letter/s/e79b87/32.png) [@ShahafSW](https://discuss.elastic.co/u/ShahafSW)\ **Post date:** [August 27, 2019, 8:40am UTC](https://discuss.elastic.co/t/migrating-elasticsearch-data/196905/1 "2019-08-27T08:40:26Z")
Hey Everyone, I have this situation that I have a disconnected elasticsearch[1] environment and I need to transfer the data to another elasticsearch[2] cluster. I was thinking to export all of the data from [1] to a Json file (with log stash pipeline) and transfer the Json file to the [2] cluster, then using the bulk api for importing the indices. Is there any better way to perform this operation given that the clusters[1]+[2] are not connected via the internet? ---
**Author:** ![wangqinghuan](https://avatars.discourse-cdn.com/v4/letter/w/d26b3c/32.png) [@wangqinghuan](https://discuss.elastic.co/u/wangqinghuan)\ **Post date:** [August 27, 2019, 8:56am UTC](https://discuss.elastic.co/t/migrating-elasticsearch-data/196905/2 "2019-08-27T08:56:53Z")
hi [Shahaf](https://discuss.elastic.co/u/ShahafSW) A more fast way is SnapShot/Restore. ---
**Author:** ![ShahafSW](https://avatars.discourse-cdn.com/v4/letter/s/e79b87/32.png) [@ShahafSW](https://discuss.elastic.co/u/ShahafSW)\ **Post date:** [August 27, 2019, 11:42am UTC](https://discuss.elastic.co/t/migrating-elasticsearch-data/196905/3 "2019-08-27T11:42:18Z")
Hey @wangqinghuan I did not mention that I am not allowed to change elasticsearch configuration (for adding "path") to the elasticsearch.yml. Therefor I am trying to export elasticsearch indices to a Json file and import it with bulk api to another cluster. This is my logstash pipeline: input { elasticsearch { hosts =\> "localhost:9200" index =\> "\*" docinfo =\> true size =\> 10000 } } output { file { path =\> "/var/log/logstash/index.json" codec =\> json\_lines } } After getting the Json file, the bulk api does not seems to work for the second cluster, am I doing anything wrong? Bulk api: curl -v -H 'Content-Type: application/x-ndjson' -XPOST 'localhost:9200/\_bulk?pretty' --data-binary @/var/log/logstash/index.json ---
**Author:** ![wangqinghuan](https://avatars.discourse-cdn.com/v4/letter/w/d26b3c/32.png) [@wangqinghuan](https://discuss.elastic.co/u/wangqinghuan)\ **Post date:** [August 28, 2019, 1:18am UTC](https://discuss.elastic.co/t/migrating-elasticsearch-data/196905/4 "2019-08-28T01:18:14Z")
Does the index.json contain action\_and\_meta\_data lines? A standard JSON structure that bulk API expects: ``` { "index" : { "_index" : "test", "_id" : "1" } } { "field1" : "value1" } ``` ---
**Author:** ![sspilleman](https://avatars.discourse-cdn.com/v4/letter/s/9dc877/32.png) [@sspilleman](https://discuss.elastic.co/u/sspilleman)\ **Post date:** [August 28, 2019, 5:31am UTC](https://discuss.elastic.co/t/migrating-elasticsearch-data/196905/5 "2019-08-28T05:31:39Z")
You could use elasticdump: [https://www.npmjs.com/package/elasticdump](https://www.npmjs.com/package/elasticdump) ---
**Author:** ![ShahafSW](https://avatars.discourse-cdn.com/v4/letter/s/e79b87/32.png) [@ShahafSW](https://discuss.elastic.co/u/ShahafSW)\ **Post date:** [August 28, 2019, 6:00am UTC](https://discuss.elastic.co/t/migrating-elasticsearch-data/196905/6 "2019-08-28T06:00:22Z")
@wangqinghuan This is probably my issue, my file does not have the first line: `{ "index" : { "_index" : "test", "_id" : "1" } }` How can I export all of my indices to one file with the structured you mentioned using logstash pipeline? @sspilleman This way requires from me configuration changes which in my case is not an option. ---
**Author:** ![wangqinghuan](https://avatars.discourse-cdn.com/v4/letter/w/d26b3c/32.png) [@wangqinghuan](https://discuss.elastic.co/u/wangqinghuan)\ **Post date:** [August 28, 2019, 8:23am UTC](https://discuss.elastic.co/t/migrating-elasticsearch-data/196905/7 "2019-08-28T08:23:02Z")
I don't know how to export meta\_data line with logstash pipeline. However, you can use logstash to import index.json into your new Elasticsearch cluster. ---
**Author:** ![ShahafSW](https://avatars.discourse-cdn.com/v4/letter/s/e79b87/32.png) [@ShahafSW](https://discuss.elastic.co/u/ShahafSW)\ **Post date:** [August 28, 2019, 9:03am UTC](https://discuss.elastic.co/t/migrating-elasticsearch-data/196905/8 "2019-08-28T09:03:07Z")
I found this python script > ``` > #!/usr/bin/env python3 > filepath = '' > metadata='{ "index": { "_index": "INDEX_NAME", "_type": "_doc" }}' > with open(filepath, mode="r",encoding="utf-8") as my_file: > for line in my_file: > print(metadata) > print(line.rstrip("\n")) > > ``` And I ran it against the exported Json I got from the first Elasticsearch cluster, this script added the missing action\_and\_meta\_data lines, then I could use the bulk api to push it to the second elasticsearch cluster. If there is a way to export the data in the following format: > ``` > { "index" : { "_index" : "test", "_id" : "1" } } > { "field1" : "value1" } > > ``` it will be the preferred way. Anyway, thanks you for your help @wangqinghuan ! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 25, 2019, 9:03am UTC](https://discuss.elastic.co/t/migrating-elasticsearch-data/196905/9 "2019-09-25T09:03:13Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.