# Use InputStream to write content when using attachment type **URL:** \ **Category:** Elasticsearch\ **Created:** [March 19, 2012, 11:05am UTC](https://discuss.elastic.co/t/use-inputstream-to-write-content-when-using-attachment-type/7049 "2012-03-19T11:05:30Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![canavar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/canavar/32/2950_2.png) [@canavar](https://discuss.elastic.co/u/canavar)\ **Post date:** [March 19, 2012, 11:05am UTC](https://discuss.elastic.co/t/use-inputstream-to-write-content-when-using-attachment-type/7049/1 "2012-03-19T11:05:30Z")
Hi, The attachment type plugin is very handy but one question raised in my head today. I want to store relatively large(~600MB) individual documents in ES. I think the only way to write the content using the Java API is using a byte array. If a byte array is used then all the content must be available in the memory. This is a serious problem for large documents. Is there a way to use an InputStream to write the content? ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [March 20, 2012, 10:03am UTC](https://discuss.elastic.co/t/use-inputstream-to-write-content-when-using-attachment-type/7049/2 "2012-03-20T10:03:42Z")
No, there isn't a way to do it. On Mon, Mar 19, 2012 at 1:05 PM, canavar [fehmican.saglam@gmail.com](mailto:fehmican.saglam@gmail.com) wrote: > Hi, > > The attachment type plugin is very handy but one question raised in my > head today. I want to store relatively large(~600MB) individual documents > in ES. I think the only way to write the content using the Java API is > using a byte array. If a byte array is used then all the content must be > available in the memory. This is a serious problem for large documents. Is > there a way to use an InputStream to write the content? ---
**Author:** ![canavar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/canavar/32/2950_2.png) [@canavar](https://discuss.elastic.co/u/canavar)\ **Post date:** [March 20, 2012, 8:30pm UTC](https://discuss.elastic.co/t/use-inputstream-to-write-content-when-using-attachment-type/7049/3 "2012-03-20T20:30:57Z")
May it be possible? Should I file an issue and try to implement it? Or do you think it is needless or hard to implement? 20 Mar 2012 12:04 tarihinde "Shay Banon" [kimchy@gmail.com](mailto:kimchy@gmail.com) yazdı: > No, there isn't a way to do it. > > On Mon, Mar 19, 2012 at 1:05 PM, canavar [fehmican.saglam@gmail.com](mailto:fehmican.saglam@gmail.com) > wrote: > > > Hi, > > The attachment type plugin is very handy but one question raised in my > > head today. I want to store relatively large(~600MB) individual documents > > in ES. I think the only way to write the content using the Java API is > > using a byte array. If a byte array is used then all the content must be > > available in the memory. This is a serious problem for large documents. Is > > there a way to use an InputStream to write the content? ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [March 25, 2012, 10:05am UTC](https://discuss.elastic.co/t/use-inputstream-to-write-content-when-using-attachment-type/7049/4 "2012-03-25T10:05:33Z")
Its actually quite complicated, since it spans multiple layers. The first is being able to somehow stream the data into the shard to index the data (and probably writing it to a temp file or something, since there is no way to stream index data). Then, we need to index that temp file into elasticsearch, and of course, of you use Tika, not sure if it supports stream parsing. On Tue, Mar 20, 2012 at 10:30 PM, canavar [fehmican.saglam@gmail.com](mailto:fehmican.saglam@gmail.com) wrote: > May it be possible? Should I file an issue and try to implement it? Or do > you think it is needless or hard to implement? > 20 Mar 2012 12:04 tarihinde "Shay Banon" [kimchy@gmail.com](mailto:kimchy@gmail.com) yazdı: > > > No, there isn't a way to do it. > > > > On Mon, Mar 19, 2012 at 1:05 PM, canavar [fehmican.saglam@gmail.com](mailto:fehmican.saglam@gmail.com) > > wrote: > > > > > Hi, > > > The attachment type plugin is very handy but one question raised in my > > > head today. I want to store relatively large(~600MB) individual documents > > > in ES. I think the only way to write the content using the Java API is > > > using a byte array. If a byte array is used then all the content must be > > > available in the memory. This is a serious problem for large documents. Is > > > there a way to use an InputStream to write the content? ---
**Author:** ![sasa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sasa/32/2165_2.png) [@sasa](https://discuss.elastic.co/u/sasa)\ **Post date:** [August 15, 2013, 9:10am UTC](https://discuss.elastic.co/t/use-inputstream-to-write-content-when-using-attachment-type/7049/5 "2013-08-15T09:10:49Z")
Tika does support streams. ES attachment type should also support it. There are number of reasons but main is optimisation of memory usage. I would like to see this happens. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:21am UTC](https://discuss.elastic.co/t/use-inputstream-to-write-content-when-using-attachment-type/7049/6 "2017-07-06T02:21:01Z")