# Attachment streaming **URL:** \ **Category:** Elasticsearch\ **Created:** [September 4, 2011, 6:59pm UTC](https://discuss.elastic.co/t/attachment-streaming/5309 "2011-09-04T18:59:25Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Kosta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kosta/32/2915_2.png) [@Kosta](https://discuss.elastic.co/u/Kosta)\ **Post date:** [September 4, 2011, 6:59pm UTC](https://discuss.elastic.co/t/attachment-streaming/5309/1 "2011-09-04T18:59:25Z")
I was wondering if there is any way to obtain an InputStream when running a get query using the Java API? The reason I am asking is because I feel that loading documents with large attachments into memory before forwarding attachments on to a 3rd party client is somewhat inefficient. (i.e. downloading via HTTP) What I would like to achieve ideally is obtain a stream to a document attachment, pass that stream to the web server so that it forwards it to the client, and base64 decode it on the fly. Intuition tells me that this should be much more efficient than fetching the entire document from ES, base64 decoding it, and then passing that in-memory byte array to the web server as a stream. Please correct me if I am missing something obvious. Thanks in advance! ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [September 5, 2011, 9:12am UTC](https://discuss.elastic.co/t/attachment-streaming/5309/2 "2011-09-05T09:12:41Z")
Thats how it works currently..., no plans to change that as its quite a hefty change (think about streaming across several nodes in the cluster), and will require a different store for attachments. On Sun, Sep 4, 2011 at 9:59 PM, Kosta [kosta.krauth@gmail.com](mailto:kosta.krauth@gmail.com) wrote: > I was wondering if there is any way to obtain an InputStream when > running a get query using the Java API? The reason I am asking is > because I feel that loading documents with large attachments into > memory before forwarding attachments on to a 3rd party client is > somewhat inefficient. (i.e. downloading via HTTP) > > What I would like to achieve ideally is obtain a stream to a document > attachment, pass that stream to the web server so that it forwards it > to the client, and base64 decode it on the fly. Intuition tells me > that this should be much more efficient than fetching the entire > document from ES, base64 decoding it, and then passing that in-memory > byte array to the web server as a stream. > > Please correct me if I am missing something obvious. Thanks in advance! ---
**Author:** ![Kosta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kosta/32/2915_2.png) [@Kosta](https://discuss.elastic.co/u/Kosta)\ **Post date:** [September 5, 2011, 8:38pm UTC](https://discuss.elastic.co/t/attachment-streaming/5309/3 "2011-09-05T20:38:18Z")
Thanks Shay! Just to clarify, when you say "that's how it works currently", which part do you mean? Automatic base64 decoding as the attachment is being streamed to the client? On Sep 5, 10:12 am, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > Thats how it works currently..., no plans to change that as its quite a > hefty change (think about streaming across several nodes in the cluster), > and will require a different store for attachments. > > On Sun, Sep 4, 2011 at 9:59 PM, Kosta [kosta.kra...@gmail.com](mailto:kosta.kra...@gmail.com) wrote: > > > I was wondering if there is any way to obtain an InputStream when > > running a get query using the Java API? The reason I am asking is > > because I feel that loading documents with large attachments into > > memory before forwarding attachments on to a 3rd party client is > > somewhat inefficient. (i.e. downloading via HTTP) > > > What I would like to achieve ideally is obtain a stream to a document > > attachment, pass that stream to the web server so that it forwards it > > to the client, and base64 decode it on the fly. Intuition tells me > > that this should be much more efficient than fetching the entire > > document from ES, base64 decoding it, and then passing that in-memory > > byte array to the web server as a stream. > > > Please correct me if I am missing something obvious. Thanks in advance! ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [September 6, 2011, 7:18am UTC](https://discuss.elastic.co/t/attachment-streaming/5309/4 "2011-09-06T07:18:15Z")
Yes, the full attachment is loaded to memory (\_source), it does not get another round of base 64 if you ask for \_source, since its stored as is (already in base64). On Mon, Sep 5, 2011 at 11:38 PM, Kosta [kosta.krauth@gmail.com](mailto:kosta.krauth@gmail.com) wrote: > Thanks Shay! Just to clarify, when you say "that's how it works > currently", which part do you mean? Automatic base64 decoding as the > attachment is being streamed to the client? > > On Sep 5, 10:12 am, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > > > Thats how it works currently..., no plans to change that as its quite a > > hefty change (think about streaming across several nodes in the cluster), > > and will require a different store for attachments. > > > > On Sun, Sep 4, 2011 at 9:59 PM, Kosta [kosta.kra...@gmail.com](mailto:kosta.kra...@gmail.com) wrote: > > > > > I was wondering if there is any way to obtain an InputStream when > > > running a get query using the Java API? The reason I am asking is > > > because I feel that loading documents with large attachments into > > > memory before forwarding attachments on to a 3rd party client is > > > somewhat inefficient. (i.e. downloading via HTTP) > > > > > What I would like to achieve ideally is obtain a stream to a document > > > attachment, pass that stream to the web server so that it forwards it > > > to the client, and base64 decode it on the fly. Intuition tells me > > > that this should be much more efficient than fetching the entire > > > document from ES, base64 decoding it, and then passing that in-memory > > > byte array to the web server as a stream. > > > > > Please correct me if I am missing something obvious. Thanks in advance! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:55am UTC](https://discuss.elastic.co/t/attachment-streaming/5309/5 "2017-07-06T03:55:11Z")