# Convert bulk request to json document and publish that document to ES as a seperate task **URL:** \ **Category:** Elasticsearch\ **Created:** [April 28, 2015, 6:21pm UTC](https://discuss.elastic.co/t/convert-bulk-request-to-json-document-and-publish-that-document-to-es-as-a-seperate-task/23448 "2015-04-28T18:21:49Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Manjula\_Piyumal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/manjula_piyumal/32/57068_2.png) [@Manjula\_Piyumal](https://discuss.elastic.co/u/Manjula_Piyumal)\ **Post date:** [April 28, 2015, 6:21pm UTC](https://discuss.elastic.co/t/convert-bulk-request-to-json-document-and-publish-that-document-to-es-as-a-seperate-task/23448/1 "2015-04-28T18:21:49Z")
Hi, I'm expecting to implement ES back statistic collecting application. For that I'm expecting to publish data using bulk API. I want to separate out request generating and request publishing tasks. First task is creating bulk request using JAVA API and write that request to a temp file as a JSON document and then request publisher publishes the requests which are in the files. I have tried to write created bulk request using writeTo method as follows. It writes the request as a semi-json document containing some random weird bytes(resulting file is attached). ``` FileOutputStream fileOutputStream = new FileOutputStream(new ``` File("request.txt")); OutputStreamStreamOutput streamOutput = new OutputStreamStreamOutput(fileOutputStream); bulkRequestBuilder.request().writeTo(streamOutput); I'm not sure whether I have missed something here. I want to understand what goes wrong while writing the request and is it possible to implement my application by separating request generating and publishing tasks as mentioned above. Any help would be appreciated. Thanks Manjula -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/690f5a0a-3c62-43f3-b662-259b4dce79e1%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/690f5a0a-3c62-43f3-b662-259b4dce79e1%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [April 28, 2015, 7:02pm UTC](https://discuss.elastic.co/t/convert-bulk-request-to-json-document-and-publish-that-document-to-es-as-a-seperate-task/23448/2 "2015-04-28T19:02:53Z")
You are using the binary stream protocol of ES in the writeTo() method which is not appropriate for writing to files. Once you added requests to a bulk request, you can not get your content back as JSON. A better approach is to use an XContentBuilder with an OutputStream, and add the content to it, independent of BulkRequestBuilder. Jörg On Tue, Apr 28, 2015 at 8:21 PM, Manjula Piyumal [manjulapiyumal@gmail.com](mailto:manjulapiyumal@gmail.com) wrote: > Hi, > > I'm expecting to implement ES back statistic collecting application. For > that I'm expecting to publish data using bulk API. I want to separate out > request generating and request publishing tasks. First task is creating > bulk request using JAVA API and write that request to a temp file as a JSON > document and then request publisher publishes the requests which are in the > files. > I have tried to write created bulk request using writeTo method as > follows. It writes the request as a semi-json document containing some > random weird bytes(resulting file is attached). > > ``` > FileOutputStream fileOutputStream = new FileOutputStream(new > > ``` > > File("request.txt")); > OutputStreamStreamOutput streamOutput = new > OutputStreamStreamOutput(fileOutputStream); > bulkRequestBuilder.request().writeTo(streamOutput); > > I'm not sure whether I have missed something here. I want to understand > what goes wrong while writing the request and is it possible to implement > my application by separating request generating and publishing tasks as > mentioned above. Any help would be appreciated. > > Thanks > Manjula > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/690f5a0a-3c62-43f3-b662-259b4dce79e1%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/690f5a0a-3c62-43f3-b662-259b4dce79e1%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/690f5a0a-3c62-43f3-b662-259b4dce79e1%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/690f5a0a-3c62-43f3-b662-259b4dce79e1%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHGAVSjOWX6%2BQ08pNUBWczf6ySYwGDm%2BXMsepMnsWysvA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHGAVSjOWX6%2BQ08pNUBWczf6ySYwGDm%2BXMsepMnsWysvA%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Manjula\_Piyumal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/manjula_piyumal/32/57068_2.png) [@Manjula\_Piyumal](https://discuss.elastic.co/u/Manjula_Piyumal)\ **Post date:** [April 28, 2015, 7:15pm UTC](https://discuss.elastic.co/t/convert-bulk-request-to-json-document-and-publish-that-document-to-es-as-a-seperate-task/23448/3 "2015-04-28T19:15:08Z")
Hi Jörg, Thanks for the quick response. I'll try it in that way. Thanks Manjula On Wed, Apr 29, 2015 at 12:32 AM, [joergprante@gmail.com](mailto:joergprante@gmail.com) \< [joergprante@gmail.com](mailto:joergprante@gmail.com)\> wrote: > You are using the binary stream protocol of ES in the writeTo() method > which is not appropriate for writing to files. > > Once you added requests to a bulk request, you can not get your content > back as JSON. > > A better approach is to use an XContentBuilder with an OutputStream, and > add the content to it, independent of BulkRequestBuilder. > > Jörg > > On Tue, Apr 28, 2015 at 8:21 PM, Manjula Piyumal \<[manjulapiyumal@gmail.com](mailto:manjulapiyumal@gmail.com) > > > wrote: > > > Hi, > > > > I'm expecting to implement ES back statistic collecting application. For > > that I'm expecting to publish data using bulk API. I want to separate out > > request generating and request publishing tasks. First task is creating > > bulk request using JAVA API and write that request to a temp file as a JSON > > document and then request publisher publishes the requests which are in the > > files. > > I have tried to write created bulk request using writeTo method as > > follows. It writes the request as a semi-json document containing some > > random weird bytes(resulting file is attached). > > > > ``` > > FileOutputStream fileOutputStream = new FileOutputStream(new > > > > ``` > > > > File("request.txt")); > > OutputStreamStreamOutput streamOutput = new > > OutputStreamStreamOutput(fileOutputStream); > > bulkRequestBuilder.request().writeTo(streamOutput); > > > > I'm not sure whether I have missed something here. I want to understand > > what goes wrong while writing the request and is it possible to implement > > my application by separating request generating and publishing tasks as > > mentioned above. Any help would be appreciated. > > > > Thanks > > Manjula > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/690f5a0a-3c62-43f3-b662-259b4dce79e1%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/690f5a0a-3c62-43f3-b662-259b4dce79e1%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/690f5a0a-3c62-43f3-b662-259b4dce79e1%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/690f5a0a-3c62-43f3-b662-259b4dce79e1%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/RCz0JgIP5kY/unsubscribe](https://groups.google.com/d/topic/elasticsearch/RCz0JgIP5kY/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHGAVSjOWX6%2BQ08pNUBWczf6ySYwGDm%2BXMsepMnsWysvA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHGAVSjOWX6%2BQ08pNUBWczf6ySYwGDm%2BXMsepMnsWysvA%40mail.gmail.com) > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHGAVSjOWX6%2BQ08pNUBWczf6ySYwGDm%2BXMsepMnsWysvA%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHGAVSjOWX6%2BQ08pNUBWczf6ySYwGDm%2BXMsepMnsWysvA%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- _Manjula Piyumal De Silva_ Software Engineer, AdroitLogic Private Ltd, -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAFLAp0XqWcdKdnyrw3vg6%2Be4tCp%3DQY2MAYDrngKfEOMTkbpkUA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAFLAp0XqWcdKdnyrw3vg6%2Be4tCp%3DQY2MAYDrngKfEOMTkbpkUA%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:17am UTC](https://discuss.elastic.co/t/convert-bulk-request-to-json-document-and-publish-that-document-to-es-as-a-seperate-task/23448/4 "2017-07-06T00:17:16Z")