# What is the best way to insert/update billion data to ES from API? **URL:** \ **Category:** Elasticsearch\ **Created:** [December 6, 2018, 1:56pm UTC](https://discuss.elastic.co/t/what-is-the-best-way-to-insert-update-billion-data-to-es-from-api/159740 "2018-12-06T13:56:53Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![inandi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/inandi/32/40801_2.png) [@inandi](https://discuss.elastic.co/u/inandi)\ **Post date:** [December 6, 2018, 1:56pm UTC](https://discuss.elastic.co/t/what-is-the-best-way-to-insert-update-billion-data-to-es-from-api/159740/1 "2018-12-06T13:56:53Z")
I am getting millions of data from Google API, Now I want to store it in ES. I know the technique to save data to ES using Logsatsh from JSON file. So should I create a JSON file and then try to save it. Or there is a better and sophisticated way like CURL? Thanks in advance 🙂 ---
**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\ **Post date:** [December 6, 2018, 3:31pm UTC](https://discuss.elastic.co/t/what-is-the-best-way-to-insert-update-billion-data-to-es-from-api/159740/2 "2018-12-06T15:31:10Z")
You might want to write an application that reads from the API and uses Elasticsearch's [`_bulk`](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html) API. You'll want some way to resume where you left of and, hopefully, some way to identify the documents that you wrote so if you accidentally get two copies of the document from a funky resume you'll just overwrite the first copy with the second. ---
**Author:** ![inandi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/inandi/32/40801_2.png) [@inandi](https://discuss.elastic.co/u/inandi)\ **Post date:** [December 7, 2018, 5:06am UTC](https://discuss.elastic.co/t/what-is-the-best-way-to-insert-update-billion-data-to-es-from-api/159740/3 "2018-12-07T05:06:05Z")
thanks @nik9000 for replying .. I will follow your advice. Peace !!! 🙂 ---
**Author:** ![inandi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/inandi/32/40801_2.png) [@inandi](https://discuss.elastic.co/u/inandi)\ **Post date:** [December 12, 2018, 10:32am UTC](https://discuss.elastic.co/t/what-is-the-best-way-to-insert-update-billion-data-to-es-from-api/159740/4 "2018-12-12T10:32:18Z")
what if I am using logstash?? ---
**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\ **Post date:** [December 12, 2018, 5:22pm UTC](https://discuss.elastic.co/t/what-is-the-best-way-to-insert-update-billion-data-to-es-from-api/159740/5 "2018-12-12T17:22:00Z")
To be honest I don't know much about logstash. I'm sure it'd work but it isn't a tool I have much experience with. If you have specific questions I'd ask them in [https://discuss.elastic.co/c/logstash](https://discuss.elastic.co/c/logstash) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 9, 2019, 5:22pm UTC](https://discuss.elastic.co/t/what-is-the-best-way-to-insert-update-billion-data-to-es-from-api/159740/6 "2019-01-09T17:22:15Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.