# Bulk http api broken? **URL:** \ **Category:** Elasticsearch\ **Created:** [April 3, 2014, 1:54pm UTC](https://discuss.elastic.co/t/bulk-http-api-broken/16791 "2014-04-03T13:54:19Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Isaac\_Dooley\_2](https://avatars.discourse-cdn.com/v4/letter/i/ebca7d/32.png) [@Isaac\_Dooley\_2](https://discuss.elastic.co/u/Isaac_Dooley_2)\ **Post date:** [April 3, 2014, 1:54pm UTC](https://discuss.elastic.co/t/bulk-http-api-broken/16791/1 "2014-04-03T13:54:19Z")
I think I've found an intermittent bug for bulk updates over HTTP in Elasticsearch. It seems to happen on versions 0.90.11, 0.90.12, 1.0.1 when running a single server instance (not a cluster) on linux. I do a bulk PUT with documents going to multiple indices (mostly new ones). In my test I then wait a few seconds and then query the mapping, and get back empty results for one of the indices. I notice in the logs that I see "creating index" messages for all the relevant indices, but the "update\_mapping" message is missing for the index that returns an empty mapping. For example here are those lines from my log: 2014-03-31 18:09:27,695 INFO [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] [mypartitioned\_20 13.11.07] creating index, cause [auto(bulk api)], shards [5]/[1], mappings [] 2014-03-31 18:09:27,940 INFO [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] [mypartitioned\_20 13.11.06] creating index, cause [auto(bulk api)], shards [5]/[1], mappings [] 2014-03-31 18:09:27,984 INFO [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] [mypartitioned\_20 13.11.05] creating index, cause [auto(bulk api)], shards [5]/[1], mappings [] 2014-03-31 18:09:28,026 INFO [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] [mypartitioned\_20 13.11.04] creating index, cause [auto(bulk api)], shards [5]/[1], mappings [] 2014-03-31 18:09:28,070 INFO [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] [mypartitioned\_20 13.11.03] creating index, cause [auto(bulk api)], shards [5]/[1], mappings [] 2014-03-31 18:09:28,112 INFO [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] [mypartitioned\_20 13.11.02] creating index, cause [auto(bulk api)], shards [5]/[1], mappings [] 2014-03-31 18:09:28,153 INFO [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] [mypartitioned\_20 13.11.01] creating index, cause [auto(bulk api)], shards [5]/[1], mappings [] 2014-03-31 18:09:28,851 INFO [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] [mypartitioned\_20 13.11.07] update\_mapping [logs] (dynamic) 2014-03-31 18:09:29,291 INFO [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] [mypartitioned\_20 13.11.06] update\_mapping [logs] (dynamic) 2014-03-31 18:09:29,297 INFO [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] [mypartitioned\_20 13.11.04] update\_mapping [logs] (dynamic) 2014-03-31 18:09:29,327 INFO [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] [mypartitioned\_20 13.11.02] update\_mapping [logs] (dynamic) 2014-03-31 18:09:29,700 INFO [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] [mypartitioned\_20 13.11.05] update\_mapping [logs] (dynamic) 2014-03-31 18:09:29,778 INFO [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] [mypartitioned\_20 13.11.03] update\_mapping [logs] (dynamic) The index with broken mapping does exist and contains data, but the \_mapping HTTP endpoint gives an empty result. Is this a known issue? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/41d1f1b5-03c4-4b9d-b02f-98104fd37a04%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/41d1f1b5-03c4-4b9d-b02f-98104fd37a04%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Britta\_Weber](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/britta_weber/32/1113_2.png) [@Britta\_Weber](https://discuss.elastic.co/u/Britta_Weber)\ **Post date:** [April 3, 2014, 2:21pm UTC](https://discuss.elastic.co/t/bulk-http-api-broken/16791/2 "2014-04-03T14:21:36Z")
Hi Isaac, I believe you ran into this bug: > > > When a bulk request triggers an index creation a mapping might not be > created. T…he reason is that when indexing documents in a bulk, > an indexing operation might fail due to a shard not yet being > started. The mapping service, however, might already > have the mapping but the mapping update is never issued to the master, > even on subsequent indexing of documents. > > Instead, the mapping must be propagated to master even if the > indexing fails due to a shard not being started. It will be fixed in upcoming releases. Britta On Thu, Apr 3, 2014 at 3:54 PM, Isaac Dooley [idooley@gmail.com](mailto:idooley@gmail.com) wrote: > I think I've found an intermittent bug for bulk updates over HTTP in > Elasticsearch. It seems to happen on versions 0.90.11, 0.90.12, 1.0.1 when > running a single server instance (not a cluster) on linux. > > I do a bulk PUT with documents going to multiple indices (mostly new ones). > In my test I then wait a few seconds and then query the mapping, and get > back empty results for one of the indices. I notice in the logs that I see > "creating index" messages for all the relevant indices, but the > "update\_mapping" message is missing for the index that returns an empty > mapping. For example here are those lines from my log: > > 2014-03-31 18:09:27,695 INFO > [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] > org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] > [mypartitioned\_20 > 13.11.07] creating index, cause [auto(bulk api)], shards [5]/[1], mappings > > 2014-03-31 18:09:27,940 INFO > [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] > org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] > [mypartitioned\_20 > 13.11.06] creating index, cause [auto(bulk api)], shards [5]/[1], mappings > > 2014-03-31 18:09:27,984 INFO > [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] > org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] > [mypartitioned\_20 > 13.11.05] creating index, cause [auto(bulk api)], shards [5]/[1], mappings > > 2014-03-31 18:09:28,026 INFO > [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] > org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] > [mypartitioned\_20 > 13.11.04] creating index, cause [auto(bulk api)], shards [5]/[1], mappings > > 2014-03-31 18:09:28,070 INFO > [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] > org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] > [mypartitioned\_20 > 13.11.03] creating index, cause [auto(bulk api)], shards [5]/[1], mappings > > 2014-03-31 18:09:28,112 INFO > [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] > org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] > [mypartitioned\_20 > 13.11.02] creating index, cause [auto(bulk api)], shards [5]/[1], mappings > > 2014-03-31 18:09:28,153 INFO > [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] > org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] > [mypartitioned\_20 > 13.11.01] creating index, cause [auto(bulk api)], shards [5]/[1], mappings > > 2014-03-31 18:09:28,851 INFO > [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] > org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] > [mypartitioned\_20 > 13.11.07] update\_mapping [logs] (dynamic) > 2014-03-31 18:09:29,291 INFO > [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] > org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] > [mypartitioned\_20 > 13.11.06] update\_mapping [logs] (dynamic) > 2014-03-31 18:09:29,297 INFO > [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] > org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] > [mypartitioned\_20 > 13.11.04] update\_mapping [logs] (dynamic) > 2014-03-31 18:09:29,327 INFO > [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] > org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] > [mypartitioned\_20 > 13.11.02] update\_mapping [logs] (dynamic) > 2014-03-31 18:09:29,700 INFO > [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] > org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] > [mypartitioned\_20 > 13.11.05] update\_mapping [logs] (dynamic) > 2014-03-31 18:09:29,778 INFO > [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] > org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] > [mypartitioned\_20 > 13.11.03] update\_mapping [logs] (dynamic) > > The index with broken mapping does exist and contains data, but the > \_mapping HTTP endpoint gives an empty result. Is this a known issue? > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/41d1f1b5-03c4-4b9d-b02f-98104fd37a04%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/41d1f1b5-03c4-4b9d-b02f-98104fd37a04%40googlegroups.com). > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CALhJbBj44riyHZLMokkoRm%3DcthHqYRJuTQNhUgzKyYKEwN3u8Q%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALhJbBj44riyHZLMokkoRm%3DcthHqYRJuTQNhUgzKyYKEwN3u8Q%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Isaac\_Dooley\_2](https://avatars.discourse-cdn.com/v4/letter/i/ebca7d/32.png) [@Isaac\_Dooley\_2](https://discuss.elastic.co/u/Isaac_Dooley_2)\ **Post date:** [April 3, 2014, 2:51pm UTC](https://discuss.elastic.co/t/bulk-http-api-broken/16791/3 "2014-04-03T14:51:51Z")
Thanks Britta, that bug description matches what I'm seeing. I'll be watching for an update (or patch a version for me to use). On Thursday, April 3, 2014 10:21:36 AM UTC-4, Britta Weber wrote: > Hi Isaac, > > I believe you ran into this bug: > [https://github.com/elasticsearch/elasticsearch/pull/5623](https://github.com/elasticsearch/elasticsearch/pull/5623) > It will be fixed in upcoming releases. > > Britta > > On Thu, Apr 3, 2014 at 3:54 PM, Isaac Dooley \<[ido...@gmail.com](mailto:ido...@gmail.com)\\> > wrote: > > > I think I've found an intermittent bug for bulk updates over HTTP in > > Elasticsearch. It seems to happen on versions 0.90.11, 0.90.12, 1.0.1 > > when > > running a single server instance (not a cluster) on linux. > > > > I do a bulk PUT with documents going to multiple indices (mostly new > > ones). > > In my test I then wait a few seconds and then query the mapping, and get > > back empty results for one of the indices. I notice in the logs that I > > see > > "creating index" messages for all the relevant indices, but the > > "update\_mapping" message is missing for the index that returns an empty > > mapping. For example here are those lines from my log: > > > > 2014-03-31 18:09:27,695 INFO > > [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] > > > org.elasticsearch.cluster.metadata > > [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] > > [mypartitioned\_20\ > > 13.11.07] creating index, cause [auto(bulk api)], shards [5]/[1], > > mappings > > > > 2014-03-31 18:09:27,940 INFO > > [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] > > > org.elasticsearch.cluster.metadata > > [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] > > [mypartitioned\_20\ > > 13.11.06] creating index, cause [auto(bulk api)], shards [5]/[1], > > mappings > > > > 2014-03-31 18:09:27,984 INFO > > [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] > > > org.elasticsearch.cluster.metadata > > [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] > > [mypartitioned\_20\ > > 13.11.05] creating index, cause [auto(bulk api)], shards [5]/[1], > > mappings > > > > 2014-03-31 18:09:28,026 INFO > > [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] > > > org.elasticsearch.cluster.metadata > > [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] > > [mypartitioned\_20\ > > 13.11.04] creating index, cause [auto(bulk api)], shards [5]/[1], > > mappings > > > > 2014-03-31 18:09:28,070 INFO > > [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] > > > org.elasticsearch.cluster.metadata > > [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] > > [mypartitioned\_20\ > > 13.11.03] creating index, cause [auto(bulk api)], shards [5]/[1], > > mappings > > > > 2014-03-31 18:09:28,112 INFO > > [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] > > > org.elasticsearch.cluster.metadata > > [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] > > [mypartitioned\_20\ > > 13.11.02] creating index, cause [auto(bulk api)], shards [5]/[1], > > mappings > > > > 2014-03-31 18:09:28,153 INFO > > [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] > > > org.elasticsearch.cluster.metadata > > [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] > > [mypartitioned\_20\ > > 13.11.01] creating index, cause [auto(bulk api)], shards [5]/[1], > > mappings > > > > 2014-03-31 18:09:28,851 INFO > > [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] > > > org.elasticsearch.cluster.metadata > > [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] > > [mypartitioned\_20\ > > 13.11.07] update\_mapping [logs] (dynamic) > > 2014-03-31 18:09:29,291 INFO > > [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] > > > org.elasticsearch.cluster.metadata > > [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] > > [mypartitioned\_20\ > > 13.11.06] update\_mapping [logs] (dynamic) > > 2014-03-31 18:09:29,297 INFO > > [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] > > > org.elasticsearch.cluster.metadata > > [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] > > [mypartitioned\_20\ > > 13.11.04] update\_mapping [logs] (dynamic) > > 2014-03-31 18:09:29,327 INFO > > [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] > > > org.elasticsearch.cluster.metadata > > [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] > > [mypartitioned\_20\ > > 13.11.02] update\_mapping [logs] (dynamic) > > 2014-03-31 18:09:29,700 INFO > > [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] > > > org.elasticsearch.cluster.metadata > > [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] > > [mypartitioned\_20\ > > 13.11.05] update\_mapping [logs] (dynamic) > > 2014-03-31 18:09:29,778 INFO > > [elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]] > > > org.elasticsearch.cluster.metadata > > [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f] > > [mypartitioned\_20\ > > 13.11.03] update\_mapping [logs] (dynamic) > > > > The index with broken mapping does exist and contains data, but the > > \_mapping HTTP endpoint gives an empty result. Is this a known issue? > > > > -- > > You received this message because you are subscribed to the Google > > Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send > > an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/41d1f1b5-03c4-4b9d-b02f-98104fd37a04%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/41d1f1b5-03c4-4b9d-b02f-98104fd37a04%40googlegroups.com). > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/b4a6e148-de3f-4279-bfa7-4442c1b9a4ee%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/b4a6e148-de3f-4279-bfa7-4442c1b9a4ee%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:38am UTC](https://discuss.elastic.co/t/bulk-http-api-broken/16791/4 "2017-07-06T01:38:32Z")