# Sporadic index corruption... Fielddata is disabled on text fields by default **URL:** \ **Category:** Elasticsearch\ **Created:** [February 27, 2017, 6:42pm UTC](https://discuss.elastic.co/t/sporadic-index-corruption-fielddata-is-disabled-on-text-fields-by-default/76679 "2017-02-27T18:42:46Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![jasona](https://avatars.discourse-cdn.com/v4/letter/j/3bc359/32.png) [@jasona](https://discuss.elastic.co/u/jasona)\ **Post date:** [February 27, 2017, 6:42pm UTC](https://discuss.elastic.co/t/sporadic-index-corruption-fielddata-is-disabled-on-text-fields-by-default/76679/1 "2017-02-27T18:42:46Z")
After the aws hardware failure, we've been encountering a new error that we've never seen before on our [elastic.co](http://elastic.co) cloud cluster. Reference: [https://discuss.elastic.co/t/problem-cluster-is-under-maintenance-recovery-for-the-last-hour/76090](https://discuss.elastic.co/t/problem-cluster-is-under-maintenance-recovery-for-the-last-hour/76090) I can blow away our mapping and template and rebuild my indexes from scratch, but then I appear to be encountering sporadic index corruption, even with the dual-data-center capability now turned on. The issue manifests itself as follows. This is on cluster, cluster ID "a02c49" as the forum topic reference above. The query: curl -XPOST '[https://user:pw@host.us-west-1.aws.found.io](https://user:pw@host.us-west-1.aws.found.io):port/silver\_jobs-\*/\_search' -d '{"query":{"bool":{"should":[{"term":{"consolidated\_status":"complete"}},{"term":{"consolidated\_status":"passed"}},{"term":{"consolidated\_status":"failed"}},{"term":{"consolidated\_status":"errored"}}],"filter":[{"range":{"at0\_creation\_time":{"gte":1487592000000,"lte":1488218400000,"format":"epoch\_millis"}}}],"minimum\_number\_should\_match":1}},"aggs":{"byDate":{"date\_histogram":{"field":"at0\_creation\_time","interval":"6h"},"aggs":{"byStatus":{"terms":{"field":"consolidated\_status"}}}}},"size":0,"sort":{"at0\_creation\_time":{"order":"desc"}}}}' The output: > { > "took": 1208, > "timed\_out": false, > "\_shards": { > "total": 23, > "successful": 13, > "failed": 10, > "failures": [ > { > "shard": 0, > "index": "silver\_jobs-2017-02-26", > "node": "jkQeLJ8VQ2WDFfSBdZpkew", > "reason": { > "type": "illegal\_argument\_exception", > "reason": "Fielddata is disabled on text fields by default. Set fielddata=true on [consolidated\_status] in order to load fieldda > ta in memory by uninverting the inverted index. Note that this can however use significant memory." > } > } > ] > }, > "hits": { > "total": 8049211, > "max\_score": 0, > "hits": > }, > "aggregations": { ... The reason this looks like sporadic corruption to me is that the field that is being complained about, `consolidated_status`, is a keyword field as defined in the mapping: > ``` > "consolidated_status": { > "type": "keyword", > "copy_to": "ft" > }, > > ``` The ft field is analyzed, but not consolidated\_status itself. I had thought that the sporadic index corruption was left behind in the 2.4.1 series, but we appear to be seeing in with the 5.1.1 Elasticsearch as well. ---
**Author:** ![jasona](https://avatars.discourse-cdn.com/v4/letter/j/3bc359/32.png) [@jasona](https://discuss.elastic.co/u/jasona)\ **Post date:** [February 27, 2017, 6:44pm UTC](https://discuss.elastic.co/t/sporadic-index-corruption-fielddata-is-disabled-on-text-fields-by-default/76679/2 "2017-02-27T18:44:45Z")
@joegallo could you take a look for me ---
**Author:** ![jasona](https://avatars.discourse-cdn.com/v4/letter/j/3bc359/32.png) [@jasona](https://discuss.elastic.co/u/jasona)\ **Post date:** [February 27, 2017, 9:50pm UTC](https://discuss.elastic.co/t/sporadic-index-corruption-fielddata-is-disabled-on-text-fields-by-default/76679/3 "2017-02-27T21:50:05Z")
if someone has email for off-forum, I can provide the fully detailed (password, exact url, etc) query that reproduces the error at will. ---
**Author:** ![jasona](https://avatars.discourse-cdn.com/v4/letter/j/3bc359/32.png) [@jasona](https://discuss.elastic.co/u/jasona)\ **Post date:** [March 1, 2017, 3:00am UTC](https://discuss.elastic.co/t/sporadic-index-corruption-fielddata-is-disabled-on-text-fields-by-default/76679/4 "2017-03-01T03:00:05Z")
Ok, I tracked this down. My bad: one template shared a name with the next, so I was blowing away a template by accident. ---
**Author:** ![jasona](https://avatars.discourse-cdn.com/v4/letter/j/3bc359/32.png) [@jasona](https://discuss.elastic.co/u/jasona)\ **Post date:** [March 1, 2017, 3:12am UTC](https://discuss.elastic.co/t/sporadic-index-corruption-fielddata-is-disabled-on-text-fields-by-default/76679/5 "2017-03-01T03:12:59Z")
The fact that elasticsearch will let you overwrite one template with another, without having to delete the first one, makes it rather error prone. Having to do two steps (delete, then write) in order to change an established template, would have caught this issue much, much sooner. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [March 29, 2017, 3:13am UTC](https://discuss.elastic.co/t/sporadic-index-corruption-fielddata-is-disabled-on-text-fields-by-default/76679/6 "2017-03-29T03:13:12Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.