Bulk http api broken?

I think I've found an intermittent bug for bulk updates over HTTP in
Elasticsearch. It seems to happen on versions 0.90.11, 0.90.12, 1.0.1 when
running a single server instance (not a cluster) on linux.

I do a bulk PUT with documents going to multiple indices (mostly new ones).
In my test I then wait a few seconds and then query the mapping, and get
back empty results for one of the indices. I notice in the logs that I see
"creating index" messages for all the relevant indices, but the
"update_mapping" message is missing for the index that returns an empty
mapping. For example here are those lines from my log:

2014-03-31 18:09:27,695 INFO
[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]
org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20
13.11.07] creating index, cause [auto(bulk api)], shards [5]/[1], mappings
[]
2014-03-31 18:09:27,940 INFO
[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]
org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20
13.11.06] creating index, cause [auto(bulk api)], shards [5]/[1], mappings
[]
2014-03-31 18:09:27,984 INFO
[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]
org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20
13.11.05] creating index, cause [auto(bulk api)], shards [5]/[1], mappings
[]
2014-03-31 18:09:28,026 INFO
[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]
org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20
13.11.04] creating index, cause [auto(bulk api)], shards [5]/[1], mappings
[]
2014-03-31 18:09:28,070 INFO
[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]
org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20
13.11.03] creating index, cause [auto(bulk api)], shards [5]/[1], mappings
[]
2014-03-31 18:09:28,112 INFO
[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]
org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20
13.11.02] creating index, cause [auto(bulk api)], shards [5]/[1], mappings
[]
2014-03-31 18:09:28,153 INFO
[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]
org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20
13.11.01] creating index, cause [auto(bulk api)], shards [5]/[1], mappings
[]
2014-03-31 18:09:28,851 INFO
[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]
org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20
13.11.07] update_mapping [logs] (dynamic)
2014-03-31 18:09:29,291 INFO
[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]
org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20
13.11.06] update_mapping [logs] (dynamic)
2014-03-31 18:09:29,297 INFO
[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]
org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20
13.11.04] update_mapping [logs] (dynamic)
2014-03-31 18:09:29,327 INFO
[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]
org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20
13.11.02] update_mapping [logs] (dynamic)
2014-03-31 18:09:29,700 INFO
[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]
org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20
13.11.05] update_mapping [logs] (dynamic)
2014-03-31 18:09:29,778 INFO
[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]
org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20
13.11.03] update_mapping [logs] (dynamic)

The index with broken mapping does exist and contains data, but the
_mapping HTTP endpoint gives an empty result. Is this a known issue?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/41d1f1b5-03c4-4b9d-b02f-98104fd37a04%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Hi Isaac,

I believe you ran into this bug:

It will be fixed in upcoming releases.

Britta

On Thu, Apr 3, 2014 at 3:54 PM, Isaac Dooley idooley@gmail.com wrote:

I think I've found an intermittent bug for bulk updates over HTTP in
Elasticsearch. It seems to happen on versions 0.90.11, 0.90.12, 1.0.1 when
running a single server instance (not a cluster) on linux.

I do a bulk PUT with documents going to multiple indices (mostly new ones).
In my test I then wait a few seconds and then query the mapping, and get
back empty results for one of the indices. I notice in the logs that I see
"creating index" messages for all the relevant indices, but the
"update_mapping" message is missing for the index that returns an empty
mapping. For example here are those lines from my log:

2014-03-31 18:09:27,695 INFO
[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]
org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20
13.11.07] creating index, cause [auto(bulk api)], shards [5]/[1], mappings

2014-03-31 18:09:27,940 INFO
[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]
org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20
13.11.06] creating index, cause [auto(bulk api)], shards [5]/[1], mappings

2014-03-31 18:09:27,984 INFO
[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]
org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20
13.11.05] creating index, cause [auto(bulk api)], shards [5]/[1], mappings

2014-03-31 18:09:28,026 INFO
[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]
org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20
13.11.04] creating index, cause [auto(bulk api)], shards [5]/[1], mappings

2014-03-31 18:09:28,070 INFO
[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]
org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20
13.11.03] creating index, cause [auto(bulk api)], shards [5]/[1], mappings

2014-03-31 18:09:28,112 INFO
[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]
org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20
13.11.02] creating index, cause [auto(bulk api)], shards [5]/[1], mappings

2014-03-31 18:09:28,153 INFO
[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]
org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20
13.11.01] creating index, cause [auto(bulk api)], shards [5]/[1], mappings

2014-03-31 18:09:28,851 INFO
[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]
org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20
13.11.07] update_mapping [logs] (dynamic)
2014-03-31 18:09:29,291 INFO
[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]
org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20
13.11.06] update_mapping [logs] (dynamic)
2014-03-31 18:09:29,297 INFO
[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]
org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20
13.11.04] update_mapping [logs] (dynamic)
2014-03-31 18:09:29,327 INFO
[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]
org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20
13.11.02] update_mapping [logs] (dynamic)
2014-03-31 18:09:29,700 INFO
[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]
org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20
13.11.05] update_mapping [logs] (dynamic)
2014-03-31 18:09:29,778 INFO
[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]
org.elasticsearch.cluster.metadata [6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20
13.11.03] update_mapping [logs] (dynamic)

The index with broken mapping does exist and contains data, but the
_mapping HTTP endpoint gives an empty result. Is this a known issue?

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/41d1f1b5-03c4-4b9d-b02f-98104fd37a04%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALhJbBj44riyHZLMokkoRm%3DcthHqYRJuTQNhUgzKyYKEwN3u8Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Thanks Britta, that bug description matches what I'm seeing. I'll be
watching for an update (or patch a version for me to use).

On Thursday, April 3, 2014 10:21:36 AM UTC-4, Britta Weber wrote:

Hi Isaac,

I believe you ran into this bug:
https://github.com/elasticsearch/elasticsearch/pull/5623
It will be fixed in upcoming releases.

Britta

On Thu, Apr 3, 2014 at 3:54 PM, Isaac Dooley <ido...@gmail.com<javascript:>>
wrote:

I think I've found an intermittent bug for bulk updates over HTTP in
Elasticsearch. It seems to happen on versions 0.90.11, 0.90.12, 1.0.1
when
running a single server instance (not a cluster) on linux.

I do a bulk PUT with documents going to multiple indices (mostly new
ones).
In my test I then wait a few seconds and then query the mapping, and get
back empty results for one of the indices. I notice in the logs that I
see
"creating index" messages for all the relevant indices, but the
"update_mapping" message is missing for the index that returns an empty
mapping. For example here are those lines from my log:

2014-03-31 18:09:27,695 INFO

[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]

org.elasticsearch.cluster.metadata
[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20\
13.11.07] creating index, cause [auto(bulk api)], shards [5]/[1],
mappings

2014-03-31 18:09:27,940 INFO

[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]

org.elasticsearch.cluster.metadata
[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20\
13.11.06] creating index, cause [auto(bulk api)], shards [5]/[1],
mappings

2014-03-31 18:09:27,984 INFO

[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]

org.elasticsearch.cluster.metadata
[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20\
13.11.05] creating index, cause [auto(bulk api)], shards [5]/[1],
mappings

2014-03-31 18:09:28,026 INFO

[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]

org.elasticsearch.cluster.metadata
[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20\
13.11.04] creating index, cause [auto(bulk api)], shards [5]/[1],
mappings

2014-03-31 18:09:28,070 INFO

[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]

org.elasticsearch.cluster.metadata
[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20\
13.11.03] creating index, cause [auto(bulk api)], shards [5]/[1],
mappings

2014-03-31 18:09:28,112 INFO

[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]

org.elasticsearch.cluster.metadata
[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20\
13.11.02] creating index, cause [auto(bulk api)], shards [5]/[1],
mappings

2014-03-31 18:09:28,153 INFO

[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]

org.elasticsearch.cluster.metadata
[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20\
13.11.01] creating index, cause [auto(bulk api)], shards [5]/[1],
mappings

2014-03-31 18:09:28,851 INFO

[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]

org.elasticsearch.cluster.metadata
[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20\
13.11.07] update_mapping [logs] (dynamic)
2014-03-31 18:09:29,291 INFO

[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]

org.elasticsearch.cluster.metadata
[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20\
13.11.06] update_mapping [logs] (dynamic)
2014-03-31 18:09:29,297 INFO

[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]

org.elasticsearch.cluster.metadata
[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20\
13.11.04] update_mapping [logs] (dynamic)
2014-03-31 18:09:29,327 INFO

[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]

org.elasticsearch.cluster.metadata
[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20\
13.11.02] update_mapping [logs] (dynamic)
2014-03-31 18:09:29,700 INFO

[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]

org.elasticsearch.cluster.metadata
[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20\
13.11.05] update_mapping [logs] (dynamic)
2014-03-31 18:09:29,778 INFO

[elasticsearch[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f][clusterService#updateTask][T#1]]

org.elasticsearch.cluster.metadata
[6fb4d619-cfa5-45ea-b1d2-e3f94c1ad23f]
[mypartitioned_20\
13.11.03] update_mapping [logs] (dynamic)

The index with broken mapping does exist and contains data, but the
_mapping HTTP endpoint gives an empty result. Is this a known issue?

--
You received this message because you are subscribed to the Google
Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit

https://groups.google.com/d/msgid/elasticsearch/41d1f1b5-03c4-4b9d-b02f-98104fd37a04%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b4a6e148-de3f-4279-bfa7-4442c1b9a4ee%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.