# Streaming from elasticsearch to local file in java **URL:** \ **Category:** Elasticsearch\ **Created:** [May 19, 2020, 1:43pm UTC](https://discuss.elastic.co/t/streaming-from-elasticsearch-to-local-file-in-java/233330 "2020-05-19T13:43:51Z")\ **Posts on this page:** 14\ **Page:** 1
**Author:** ![Pankaj\_Kumar1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pankaj_kumar1/32/49853_2.png) [@Pankaj\_Kumar1](https://discuss.elastic.co/u/Pankaj_Kumar1)\ **Post date:** [May 19, 2020, 1:43pm UTC](https://discuss.elastic.co/t/streaming-from-elasticsearch-to-local-file-in-java/233330/1 "2020-05-19T13:43:51Z")
I have a large Document in elasticsearch(deployed in server) and i want get that document and store in local file without give more load on JVM- using streaming. **Is it doable or not** , if yes how? for example- in elasticsearch, Document size is 1GB and i have 500MB free memory available in JVM and i want few MB to be used out of 500MB in process. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [May 19, 2020, 1:50pm UTC](https://discuss.elastic.co/t/streaming-from-elasticsearch-to-local-file-in-java/233330/2 "2020-05-19T13:50:24Z")
Elasticsearch is not designed or optimised for documents that large so I do not think your scenario is possible/realistic. ---
**Author:** ![Pankaj\_Kumar1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pankaj_kumar1/32/49853_2.png) [@Pankaj\_Kumar1](https://discuss.elastic.co/u/Pankaj_Kumar1)\ **Post date:** [May 19, 2020, 2:21pm UTC](https://discuss.elastic.co/t/streaming-from-elasticsearch-to-local-file-in-java/233330/3 "2020-05-19T14:21:19Z")
Appreciated much, this is the current scenario i am in which so, Still searching ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [May 19, 2020, 2:24pm UTC](https://discuss.elastic.co/t/streaming-from-elasticsearch-to-local-file-in-java/233330/4 "2020-05-19T14:24:18Z")
Do you really have documents if that size? If so, why? ---
**Author:** ![Pankaj\_Kumar1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pankaj_kumar1/32/49853_2.png) [@Pankaj\_Kumar1](https://discuss.elastic.co/u/Pankaj_Kumar1)\ **Post date:** [May 19, 2020, 2:30pm UTC](https://discuss.elastic.co/t/streaming-from-elasticsearch-to-local-file-in-java/233330/5 "2020-05-19T14:30:05Z")
just for example i have mentioned, but i will be having around 100 MB, but still i can not give load JVM, so i will allocate let say 1 to 10 MB for this process(may be some bucket size of buffer size as per solution) ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [May 19, 2020, 2:37pm UTC](https://discuss.elastic.co/t/streaming-from-elasticsearch-to-local-file-in-java/233330/6 "2020-05-19T14:37:50Z")
There is no way to stream individual documents so having large documents like this requires extra heap. This is why it is not recommended. What makes these documents so large? ---
**Author:** ![Pankaj\_Kumar1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pankaj_kumar1/32/49853_2.png) [@Pankaj\_Kumar1](https://discuss.elastic.co/u/Pankaj_Kumar1)\ **Post date:** [May 19, 2020, 3:13pm UTC](https://discuss.elastic.co/t/streaming-from-elasticsearch-to-local-file-in-java/233330/7 "2020-05-19T15:13:44Z")
just json data only ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [May 19, 2020, 4:03pm UTC](https://discuss.elastic.co/t/streaming-from-elasticsearch-to-local-file-in-java/233330/8 "2020-05-19T16:03:06Z")
Just text? No encoded binary data that inflates size? ---
**Author:** ![Pankaj\_Kumar1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pankaj_kumar1/32/49853_2.png) [@Pankaj\_Kumar1](https://discuss.elastic.co/u/Pankaj_Kumar1)\ **Post date:** [May 19, 2020, 5:09pm UTC](https://discuss.elastic.co/t/streaming-from-elasticsearch-to-local-file-in-java/233330/9 "2020-05-19T17:09:21Z")
yes, just text data ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [May 19, 2020, 5:31pm UTC](https://discuss.elastic.co/t/streaming-from-elasticsearch-to-local-file-in-java/233330/10 "2020-05-19T17:31:32Z")
Could you share an example of a typical document you want to index? You can share on [gist.github.com](http://gist.github.com). ---
**Author:** ![Pankaj\_Kumar1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pankaj_kumar1/32/49853_2.png) [@Pankaj\_Kumar1](https://discuss.elastic.co/u/Pankaj_Kumar1)\ **Post date:** [May 19, 2020, 6:24pm UTC](https://discuss.elastic.co/t/streaming-from-elasticsearch-to-local-file-in-java/233330/11 "2020-05-19T18:24:25Z")
Thanks for your response, any sample Josn data you can assume/construct by repeating same data you can make 500000 lines ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [May 19, 2020, 6:41pm UTC](https://discuss.elastic.co/t/streaming-from-elasticsearch-to-local-file-in-java/233330/12 "2020-05-19T18:41:26Z")
What kind of data is this? Can you break it up somehow? What do you do with the result once you find these documents in a search? ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [May 19, 2020, 11:42pm UTC](https://discuss.elastic.co/t/streaming-from-elasticsearch-to-local-file-in-java/233330/13 "2020-05-19T23:42:31Z")
That is not going to help if I can't see your typical JSON. I'd love to help and make suggestions but I can't without any idea of what you are doing. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 16, 2020, 11:42pm UTC](https://discuss.elastic.co/t/streaming-from-elasticsearch-to-local-file-in-java/233330/14 "2020-06-16T23:42:36Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.