# Best way to export Index data(10 million) from elastic search to custom json format

**URL:** <https://discuss.elastic.co/t/best-way-to-export-index-data-10-million-from-elastic-search-to-custom-json-format/188265>\
**Category:** Elasticsearch\
**Created:** [July 1, 2019, 8:57am UTC](https://discuss.elastic.co/t/best-way-to-export-index-data-10-million-from-elastic-search-to-custom-json-format/188265 "2019-07-01T08:57:22Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![SudevPK](https://avatars.discourse-cdn.com/v4/letter/s/b5e925/32.png) [@SudevPK](https://discuss.elastic.co/u/SudevPK)\
**Post date:** [July 1, 2019, 8:57am UTC](https://discuss.elastic.co/t/best-way-to-export-index-data-10-million-from-elastic-search-to-custom-json-format/188265/1 "2019-07-01T08:57:22Z")

</div>

**Sample index data**

{  
"\_index" : "sampleindex",  
"\_type" : "\_doc",  
"\_id" : "196575730",  
"\_version" : 2,  
"\_seq\_no" : 2230396,  
"\_primary\_term" : 1,  
"found" : true,  
"\_source" : {  
"id" : 19657,  
"name" : "wer\_1212",  
"price" : "1000.0",  
"updatedDate" : "2019-06-26T15:30:42.824993-05:00"  
}  
}

**Expected Output format**

id|name|price|updateddate  
19657|wer\_1212|1000.0|06/26/2019

---

<div class="post-metadata">

**Author:** ![Miguel1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/miguel1/32/82094_2.png) [@Miguel1](https://discuss.elastic.co/u/Miguel1)\
**Post date:** [July 1, 2019, 9:07am UTC](https://discuss.elastic.co/t/best-way-to-export-index-data-10-million-from-elastic-search-to-custom-json-format/188265/2 "2019-07-01T09:07:03Z")

</div>

I would go with one of this two options, the one you feel more confortable with:  
Just with Logstash:  
[elasticsearch-input](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-elasticsearch.html#plugins-inputs-elasticsearch-scroll) using scroll option and [output-file](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-file.html)  
The other option is to use a client library like Python. It provides you an iterator over your query, so it is very memory efficient:  
[doc-here](https://elasticsearch-py.readthedocs.io/en/master/helpers.html)

---

<div class="post-metadata">

**Author:** ![SudevPK](https://avatars.discourse-cdn.com/v4/letter/s/b5e925/32.png) [@SudevPK](https://discuss.elastic.co/u/SudevPK)\
**Post date:** [July 1, 2019, 9:14am UTC](https://discuss.elastic.co/t/best-way-to-export-index-data-10-million-from-elastic-search-to-custom-json-format/188265/3 "2019-07-01T09:14:01Z")

</div>

Thanks @Miguel1

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 29, 2019, 9:14am UTC](https://discuss.elastic.co/t/best-way-to-export-index-data-10-million-from-elastic-search-to-custom-json-format/188265/4 "2019-07-29T09:14:07Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
