# Saving Results from Millions of Documents of varying sizes with Python Elasticsearch Client
**URL:** https://discuss.elastic.co/t/saving-results-from-millions-of-documents-of-varying-sizes-with-python-elasticsearch-client/80421
**Category:** Elasticsearch
**Created:** [March 29, 2017, 7:20am UTC](https://discuss.elastic.co/t/saving-results-from-millions-of-documents-of-varying-sizes-with-python-elasticsearch-client/80421 "2017-03-29T07:20:58Z")
**Posts on this page:** 2
**Page:** 1
### Author:  [@hacker\_21](https://discuss.elastic.co/u/hacker_21)
#### Post date: [March 29, 2017, 7:20am UTC](https://discuss.elastic.co/t/saving-results-from-millions-of-documents-of-varying-sizes-with-python-elasticsearch-client/80421/1 "2017-03-29T07:20:58Z")
Hi everyone,
I'm trying to get feedback on the best approach for querying my elasticsearch cluster, I have:
- 6 billion documents in one index
- documents of varying sizes, of the format:
{
"domain" : "[website1.com](http://website1.com)",
"links" : ["[link1.com](http://link1.com)", "[link2.com](http://link2.com)", "[link3.com](http://link3.com)"]
}
- each document can have as many items as possible in the "links" list ... my estimate is anywhere from 1 to 10's of thousands items
- a small search result for a query might return 3 documents, where as a large one might return 40 million documents
Ideally, I want my script to be able to export millions of document results from elasticsearch and save them as one or multiple plain text, CSV, or JSON files. What would be a fast and ideally cost-effective way of querying my database and saving the results? Are there any best practices?
Here's what I've come up with so far:
1. One machine with lots of memory that uses the scan helper function going a maximum of 10-20k documents size per request?
2. Some kind of distributed approach which queries a document range amongst 7 different machines. So, it would calculate the total # of document results and then divide querying them 1,000 at a time in some kind of cluster that saves the results.
3. ??? another option which I'm missing
Any help would be greatly appreciated, thank you in advance!
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [April 26, 2017, 7:21am UTC](https://discuss.elastic.co/t/saving-results-from-millions-of-documents-of-varying-sizes-with-python-elasticsearch-client/80421/2 "2017-04-26T07:21:01Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.