# Index corruption **URL:** \ **Category:** Elasticsearch\ **Created:** [September 30, 2011, 7:08pm UTC](https://discuss.elastic.co/t/index-corruption/5492 "2011-09-30T19:08:43Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![powellke](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/powellke/32/3103_2.png) [@powellke](https://discuss.elastic.co/u/powellke)\ **Post date:** [September 30, 2011, 7:08pm UTC](https://discuss.elastic.co/t/index-corruption/5492/1 "2011-09-30T19:08:43Z")
From time to time, my index gets corrupt and no longer returns the proper facet information. I am running version 0.17.6 on Windows 7 64-bit. In order to get my facet information to work the way I need it, I have created a default-mapping.json file in the config folder with this in it: { "record": { "dynamic\_templates": [{ "index\_nonanalyzed": { "match": "\*", "match\_mapping\_type": "string", "mapping": { "type": "string", "index": "not\_analyzed" } } }] } } I don't want any of my strings to be analyzed so they will be returned "intact". For example, I want my facets to be like this "dirt and sand" instead of it being broke up into three facets of "dirt", "and", and "sand". I'm not sure if the dynamic templates approach is the best way to achieve this but it seems to work. However, when I stop ES and restart it by running elasticsearch.bat, I've noticed that as long as I see the output say "re-syncing mappings with cluster state for type [[record]]", the facets remain correct. If it doesn't output that but only says "started", the facets return "exploded" into separate pieces. Once this happens, it never goes back to having intact facets; even if it says "re-syncing mappings...". The only way to recover is to delete my data directory and reindex everything. Here are what the facets should look like: [https://lh5.googleusercontent.com/-oSIuDNVGgn8/ToYThMoWJdI/AAAAAAAAAY0/1n7MayzKC20/s400/facets\_good.png](https://lh5.googleusercontent.com/-oSIuDNVGgn8/ToYThMoWJdI/AAAAAAAAAY0/1n7MayzKC20/s400/facets_good.png) Here they are when they get messed up: [https://lh4.googleusercontent.com/-wwzbJrmyYGg/ToYToWrsaNI/AAAAAAAAAY4/WNbOzZrEqTQ/s400/facets\_bad.png](https://lh4.googleusercontent.com/-wwzbJrmyYGg/ToYToWrsaNI/AAAAAAAAAY4/WNbOzZrEqTQ/s400/facets_bad.png) Does anyone have any insight on what could be going on? Thanks. ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [October 2, 2011, 1:07pm UTC](https://discuss.elastic.co/t/index-corruption/5492/2 "2011-10-02T13:07:22Z")
How many nodes do you have in the cluster? Also note that the default-mapping config is wrong, the mapping type should not be "record", it should be "_default_", but it might work for your case. Its not really an index corruption, but it sounds like maybe something gets wrong with the mappings, and replaying the transaction log on restart. Can you create a simple recreation of this. Something like: Place this mapping in default-mapping.json, start X nodes, run this curl requests, shutdown the nodes, start them back up, and see the problem. On Fri, Sep 30, 2011 at 10:08 PM, Keith Powell [powellke@gmail.com](mailto:powellke@gmail.com) wrote: > From time to time, my index gets corrupt and no longer returns the proper > facet information. I am running version 0.17.6 on Windows 7 64-bit. In order > to get my facet information to work the way I need it, I have created a > default-mapping.json file in the config folder with this in it: > > { > "record": { > "dynamic\_templates": [{ > "index\_nonanalyzed": { > "match": "\*", > "match\_mapping\_type": "string", > "mapping": { > "type": "string", > "index": "not\_analyzed" > } > } > }] > } > } > > I don't want any of my strings to be analyzed so they will be returned > "intact". For example, I want my facets to be like this "dirt and sand" > instead of it being broke up into three facets of "dirt", "and", and "sand". > I'm not sure if the dynamic templates approach is the best way to achieve > this but it seems to work. > > However, when I stop ES and restart it by running elasticsearch.bat, I've > noticed that as long as I see the output say "re-syncing mappings with > cluster state for type [[record]]", the facets remain correct. If it doesn't > output that but only says "started", the facets return "exploded" into > separate pieces. Once this happens, it never goes back to having intact > facets; even if it says "re-syncing mappings...". > > The only way to recover is to delete my data directory and reindex > everything. > > Here are what the facets should look like: > > [https://lh5.googleusercontent.com/-oSIuDNVGgn8/ToYThMoWJdI/AAAAAAAAAY0/1n7MayzKC20/s400/facets\_good.png](https://lh5.googleusercontent.com/-oSIuDNVGgn8/ToYThMoWJdI/AAAAAAAAAY0/1n7MayzKC20/s400/facets_good.png) > > Here they are when they get messed up: > > [https://lh4.googleusercontent.com/-wwzbJrmyYGg/ToYToWrsaNI/AAAAAAAAAY4/WNbOzZrEqTQ/s400/facets\_bad.png](https://lh4.googleusercontent.com/-wwzbJrmyYGg/ToYToWrsaNI/AAAAAAAAAY4/WNbOzZrEqTQ/s400/facets_bad.png) > > Does anyone have any insight on what could be going on? > > Thanks. ---
**Author:** ![powellke](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/powellke/32/3103_2.png) [@powellke](https://discuss.elastic.co/u/powellke)\ **Post date:** [October 3, 2011, 11:45pm UTC](https://discuss.elastic.co/t/index-corruption/5492/3 "2011-10-03T23:45:20Z")
I have using only 1 node. I made the change to the default mapping config. I thought I could specify my type in the default mapping since it's the only one where I don't want strings analyzed. I haven't been able to reproduce this issue today. It's possible it only occurs after it runs for a longer period of time. I'll see what happens tomorrow. I just switched to version 0.17.7 and noticed that Ctrl+C to terminate the batch file is working again. Could the problem be related to ES not shutting down gracefully when I simply closed the command prompt window before? ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [October 4, 2011, 11:07am UTC](https://discuss.elastic.co/t/index-corruption/5492/4 "2011-10-04T11:07:40Z")
> I just switched to version 0.17.7 and noticed that Ctrl+C to terminate the > batch file is working again. Could the problem be related to ES not shutting > down gracefully when I simply closed the command prompt window before? No, elasticsearch support a "kill -9" behavior and you don't have to shut it down gracefully. ---
**Author:** ![powellke](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/powellke/32/3103_2.png) [@powellke](https://discuss.elastic.co/u/powellke)\ **Post date:** [October 18, 2011, 5:48pm UTC](https://discuss.elastic.co/t/index-corruption/5492/5 "2011-10-18T17:48:03Z")
I haven't been able to reliably recreate this situation but it happened again this morning. Last night, I put my laptop to sleep (with ES still running). When I got to work, I docked my laptop, started it, and it locked up. The lockup isn't ES related... it just locks up from time to time when I put it in the docking station. Anyway, after I forced a restart of the computer, I restarted ES and noticed that it was showing records that I had deleted a few days ago and the facets were messed up as in the screenshots I posted earlier. I used ElasticSearch Head ([http://mobz.github.com/elasticsearch-head/](http://mobz.github.com/elasticsearch-head/)) to look at it and it showed two nodes running but with different IP address; my current IP address and my previous IP address. I typically only run one node. Does any of this make sense? Luckily, we haven't seen this issue on our production server just my dev laptop. Thanks. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:51am UTC](https://discuss.elastic.co/t/index-corruption/5492/6 "2017-07-06T03:51:31Z")