# Bulk UDP API **URL:** \ **Category:** Elasticsearch\ **Created:** [August 29, 2014, 9:36am UTC](https://discuss.elastic.co/t/bulk-udp-api/19525 "2014-08-29T09:36:44Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Bart\_Vandewoestyne](https://avatars.discourse-cdn.com/v4/letter/b/258eb7/32.png) [@Bart\_Vandewoestyne](https://discuss.elastic.co/u/Bart_Vandewoestyne)\ **Post date:** [August 29, 2014, 9:36am UTC](https://discuss.elastic.co/t/bulk-udp-api/19525/1 "2014-08-29T09:36:44Z")
I'm trying to index data using the bulk UDP API on a single node Elasticsearch 1.3.2. In my elasticsearch config I have bulk.udp.enabled: true My bulk file has 85000 documents and has the following characteristics: bart@hp-g7-02:~/git/data$ ls -al mydata.json -rw-rw-r-- 1 bart bart 97818287 Aug 28 15:43 mydata.json bart@hp-g7-02:~/git/data$ wc -l mydata.json 170001 mydata.json bart@hp-g7-02:~/git/data$ file mydata.json mydata.json: UTF-8 Unicode English text, with very long lines Indexing the data using the bulk API described at [http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-bulk.html](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-bulk.html) works. I see the documents in my elasticsearch store once the bulk upload is finished. However, if I use the same bulk file and try to index it using the command cat mydata.json | nc -w 0 -u localhost 9700 then only 1 document gets indexed, and I see lots of parsing errors like the following in my log files: [2014-08-29 11:28:41,649][WARN][bulk.udp] [Mysterio] failed to execute bulk request org.elasticsearch.common.jackson.core.JsonParseException: Unrecognized token '\_index': was expecting ('true', 'false' or 'null') at [Source: [B@656f95ce; line: 1, column: 15] at org.elasticsearch.common.jackson.core.JsonParser.\_constructError( JsonParser.java:1419) at org.elasticsearch.common.jackson.core.base.ParserMinimalBase. \_reportError(ParserMinimalBase.java:508) at org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser. \_reportInvalidToken(UTF8StreamJsonParser.java:3201) at org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser. \_handleUnexpectedValue(UTF8StreamJsonParser.java:2360) at org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser. \_nextTokenNotInObject(UTF8StreamJsonParser.java:794) at org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser. nextToken(UTF8StreamJsonParser.java:690) at org.elasticsearch.common.xcontent.json.JsonXContentParser. nextToken(JsonXContentParser.java:50) at org.elasticsearch.action.bulk.BulkRequest.add(BulkRequest.java: 266) at org.elasticsearch.action.bulk.BulkProcessor.add(BulkProcessor. java:256) at org.elasticsearch.action.bulk.BulkProcessor.add(BulkProcessor. java:252) at org.elasticsearch.bulk.udp.BulkUdpService$Handler.messageReceived (BulkUdpService.java:181) at org.elasticsearch.common.netty.channel. SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler. java:70) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline. sendUpstream(DefaultChannelPipeline.java:564) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline. sendUpstream(DefaultChannelPipeline.java:559) at org.elasticsearch.common.netty.channel.Channels. fireMessageReceived(Channels.java:268) at org.elasticsearch.common.netty.channel.socket.nio. NioDatagramWorker.read(NioDatagramWorker.java:98) at org.elasticsearch.common.netty.channel.socket.nio. AbstractNioWorker.process(AbstractNioWorker.java:108) at org.elasticsearch.common.netty.channel.socket.nio. AbstractNioSelector.run(AbstractNioSelector.java:318) at org.elasticsearch.common.netty.channel.socket.nio. AbstractNioWorker.run(AbstractNioWorker.java:89) at org.elasticsearch.common.netty.channel.socket.nio. NioDatagramWorker.run(NioDatagramWorker.java:343) at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run( ThreadRenamingRunnable.java:108) at org.elasticsearch.common.netty.util.internal. DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) at java.util.concurrent.ThreadPoolExecutor.runWorker( ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run( ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) I find it strange that things work using the usual bulk API, but not with the bulk UDP API. Am I overlooking something or doing something wrong? Thanks, Bart -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/6a676c4f-afd1-48a1-ab40-8c258aa3c54e%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6a676c4f-afd1-48a1-ab40-8c258aa3c54e%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [August 29, 2014, 12:54pm UTC](https://discuss.elastic.co/t/bulk-udp-api/19525/2 "2014-08-29T12:54:20Z")
Maybe it is the line feeds in mydata.json, probably you are not using UNIX LFs with single \n ? Jörg On Fri, Aug 29, 2014 at 11:36 AM, Bart Vandewoestyne \< [bart.vandewoestyne@gmail.com](mailto:bart.vandewoestyne@gmail.com)\> wrote: > I'm trying to index data using the bulk UDP API on a single node > Elasticsearch 1.3.2. In my elasticsearch config I have > > bulk.udp.enabled: true > > My bulk file has 85000 documents and has the following characteristics: > > bart@hp-g7-02:~/git/data$ ls -al mydata.json > -rw-rw-r-- 1 bart bart 97818287 Aug 28 15:43 mydata.json > > bart@hp-g7-02:~/git/data$ wc -l mydata.json > 170001 mydata.json > > bart@hp-g7-02:~/git/data$ file mydata.json > mydata.json: UTF-8 Unicode English text, with very long lines > > Indexing the data using the bulk API described at > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-bulk.html) > works. I see the documents in my elasticsearch store once the bulk upload > is finished. > > However, if I use the same bulk file and try to index it using the command > > cat mydata.json | nc -w 0 -u localhost 9700 > > then only 1 document gets indexed, and I see lots of parsing errors like > the following in my log files: > > [2014-08-29 11:28:41,649][WARN][bulk.udp] [Mysterio] > failed to execute bulk request > org.elasticsearch.common.jackson.core.JsonParseException: Unrecognized > token '\_index': was expecting ('true', 'false' or 'null') > at [Source: [B@656f95ce; line: 1, column: 15] > at org.elasticsearch.common.jackson.core.JsonParser. > \_constructError(JsonParser.java:1419) > at org.elasticsearch.common.jackson.core.base.ParserMinimalBase. > \_reportError(ParserMinimalBase.java:508) > at org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser > .\_reportInvalidToken(UTF8StreamJsonParser.java:3201) > at org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser > .\_handleUnexpectedValue(UTF8StreamJsonParser.java:2360) > at org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser > .\_nextTokenNotInObject(UTF8StreamJsonParser.java:794) > at org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser > .nextToken(UTF8StreamJsonParser.java:690) > at org.elasticsearch.common.xcontent.json.JsonXContentParser. > nextToken(JsonXContentParser.java:50) > at org.elasticsearch.action.bulk.BulkRequest.add(BulkRequest.java: > 266) > at org.elasticsearch.action.bulk.BulkProcessor.add(BulkProcessor. > java:256) > at org.elasticsearch.action.bulk.BulkProcessor.add(BulkProcessor. > java:252) > at org.elasticsearch.bulk.udp.BulkUdpService$Handler. > messageReceived(BulkUdpService.java:181) > at org.elasticsearch.common.netty.channel. > SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler. > java:70) > at org.elasticsearch.common.netty.channel.DefaultChannelPipeline. > sendUpstream(DefaultChannelPipeline.java:564) > at org.elasticsearch.common.netty.channel.DefaultChannelPipeline. > sendUpstream(DefaultChannelPipeline.java:559) > at org.elasticsearch.common.netty.channel.Channels. > fireMessageReceived(Channels.java:268) > at org.elasticsearch.common.netty.channel.socket.nio. > NioDatagramWorker.read(NioDatagramWorker.java:98) > at org.elasticsearch.common.netty.channel.socket.nio. > AbstractNioWorker.process(AbstractNioWorker.java:108) > at org.elasticsearch.common.netty.channel.socket.nio. > AbstractNioSelector.run(AbstractNioSelector.java:318) > at org.elasticsearch.common.netty.channel.socket.nio. > AbstractNioWorker.run(AbstractNioWorker.java:89) > at org.elasticsearch.common.netty.channel.socket.nio. > NioDatagramWorker.run(NioDatagramWorker.java:343) > at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run( > ThreadRenamingRunnable.java:108) > at org.elasticsearch.common.netty.util.internal. > DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) > at java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1145) > at java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > > I find it strange that things work using the usual bulk API, but not with > the bulk UDP API. > > Am I overlooking something or doing something wrong? > > Thanks, > Bart > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/6a676c4f-afd1-48a1-ab40-8c258aa3c54e%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6a676c4f-afd1-48a1-ab40-8c258aa3c54e%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/6a676c4f-afd1-48a1-ab40-8c258aa3c54e%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/6a676c4f-afd1-48a1-ab40-8c258aa3c54e%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGEwoT7%3Di%2BkMrTWg%2BXaeELNrWJEhdvU0h9JZAd9ocANeA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGEwoT7%3Di%2BkMrTWg%2BXaeELNrWJEhdvU0h9JZAd9ocANeA%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Bart\_Vandewoestyne](https://avatars.discourse-cdn.com/v4/letter/b/258eb7/32.png) [@Bart\_Vandewoestyne](https://discuss.elastic.co/u/Bart_Vandewoestyne)\ **Post date:** [August 29, 2014, 2:09pm UTC](https://discuss.elastic.co/t/bulk-udp-api/19525/3 "2014-08-29T14:09:50Z")
On Friday, August 29, 2014 2:54:27 PM UTC+2, Jörg Prante wrote: > Maybe it is the line feeds in mydata.json, probably you are not using UNIX > LFs with single \n ? > > Jörg I'm quite sure my data file has only UNIX LFs with a single \n: $ egrep $'\r'$ mydata.json $ and also, applying fromdos does not change the file: $ md5sum mydata.json bfd17fdea28da79152965455b594b6fe mydata.json $ fromdos mydata.json $ md5sum mydata.json bfd17fdea28da79152965455b594b6fe mydata.json So this is probably not the cause of the problem... Regards, Bart -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/67545c8b-309d-422f-9eea-ea8141b18aac%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/67545c8b-309d-422f-9eea-ea8141b18aac%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:05am UTC](https://discuss.elastic.co/t/bulk-udp-api/19525/4 "2017-07-06T01:05:30Z")