Search operation fails after restoring a snapshot

@Luca_Belluccini - Wanted to record my current status with this issue. Made progress using commands suggested by you and also @oneoneonepig.

My feeling now is that there is lack of incompatibility between snapshot (created from Linux) and my Windows environment, despite the same version V7.3.2 in both places.

Restore step does not show any problem in Kibana, but Elasticsearch log shows this -
Failed to parse mapping [xyz]: Root mapping definition has unsupported parameters: [_size : {enabled=true}]

Below, I am reporting the steps for record purpose.
I have modified repo name, index name and the mapping name for reasons of confidentiality.

I have used - repo name=myrepo, index=myindex, mapping=xyz.

  1. First made clean installation. Could add some data, by using "accounts.json" and created index "bank". Overall status is green.

  2. Changed YML and restarted for restoring snapshot obtained from external source.

path.repo: ["C:\\Elasticsearch\\myrepo"]
  1. Register REPO as readonly
PUT /_snapshot/myrepo
{
  "type" : "fs",
  "settings" : {
    "readonly" : true,
    "location" : "C:\\Elasticsearch\\myrepo"
  }
}
  1. List repositories - works fine
GET /_cat/repositories
  1. List snapshots in that repository
GET /_cat/snapshots/myrepo?v&s=id

snapshot_1 SUCCESS 1578424368 19:12:48 1578424368 19:12:48 467ms 1 5 0 5
  1. Restore snapshot
POST /_snapshot/myrepo/snapshot_1/_restore
{
  "indices": "myindex",
  "ignore_unavailable": true,
  "include_global_state": false
}
  1. This is when the problem starts - status turns red.
GET /_cat/indices (shows the issue of "myindex")

green open .kibana_task_manager       swM7-6kSQxqTceT8rpA63w 1 0    2 0  45.5kb  45.5kb
green open bank                       dyXgSbtZTLqZGOWD07kgsQ 1 0 1000 0 427.9kb 427.9kb
red   open myindex                    ZCD6JcxcTdyACRj0f8OiJw 5 0                       
green open .kibana_1                  6VOkG87tTz2FwO_4sYfN0A 1 0    4 1  23.9kb  23.9kb
  1. Failure is visible from Elasticsearch log file
[2020-04-12T10:55:44,016][INFO ][o.e.r.RepositoriesService] [CDDMLW008] put repository [myrepo]
[2020-04-12T10:58:32,245][WARN ][o.e.i.m.MapperService    ] [CDDMLW008] [myindex] [[myindex/ZCD6JcxcTdyACRj0f8OiJw]] failed to apply mappings
org.elasticsearch.index.mapper.MapperParsingException: Failed to parse mapping [xyz]: Root mapping definition has unsupported parameters:  [_size : {enabled=true}]
        at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:402) ~[elasticsearch-7.3.2.jar:7.3.2]
        at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:356) ~[elasticsearch-7.3.2.jar:7.3.2]
        at org.elasticsearch.index.mapper.MapperService.updateMapping(MapperService.java:218) [elasticsearch-7.3.2.jar:7.3.2]
        at org.elasticsearch.index.IndexService.updateMapping(IndexService.java:581) [elasticsearch-7.3.2.jar:7.3.2]
        at org.elasticsearch.indices.cluster.IndicesClusterStateService.createIndices(IndicesClusterStateService.java:504) [elasticsearch-7.3.2.jar:7.3.2]
        at org.elasticsearch.indices.cluster.IndicesClusterStateService.applyClusterState(IndicesClusterStateService.java:265) [elasticsearch-7.3.2.jar:7.3.2]
        at org.elasticsearch.cluster.service.ClusterApplierService.lambda$callClusterStateAppliers$5(ClusterApplierService.java:495) [elasticsearch-7.3.2.jar:7.3.2]
        at java.lang.Iterable.forEach(Iterable.java:75) [?:?]
        at org.elasticsearch.cluster.service.ClusterApplierService.callClusterStateAppliers(ClusterApplierService.java:493) [elasticsearch-7.3.2.jar:7.3.2]
        at org.elasticsearch.cluster.service.ClusterApplierService.applyChanges(ClusterApplierService.java:464) [elasticsearch-7.3.2.jar:7.3.2]
        at org.elasticsearch.cluster.service.ClusterApplierService.runTask(ClusterApplierService.java:418) [elasticsearch-7.3.2.jar:7.3.2]
        at org.elasticsearch.cluster.service.ClusterApplierService$UpdateTask.run(ClusterApplierService.java:165) [elasticsearch-7.3.2.jar:7.3.2]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:688) [elasticsearch-7.3.2.jar:7.3.2]
        at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:252) 
[elasticsearch-7.3.2.jar:7.3.2]
        at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:215) 
[elasticsearch-7.3.2.jar:7.3.2]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
        at java.lang.Thread.run(Thread.java:835) [?:?]
Caused by: org.elasticsearch.index.mapper.MapperParsingException: Root mapping definition has unsupported parameters:  [_size : {enabled=true}]
        at org.elasticsearch.index.mapper.DocumentMapperParser.checkNoRemainingFields(DocumentMapperParser.java:148) ~[elasticsearch-7.3.2.jar:7.3.2]
        at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:136) ~[elasticsearch-7.3.2.jar:7.3.2]
        at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:88) ~[elasticsearch-7.3.2.jar:7.3.2]
        at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:400) ~[elasticsearch-7.3.2.jar:7.3.2]
        ... 17 more
[2020-04-12T10:58:32,298][WARN ][o.e.i.c.IndicesClusterStateService] [CDDMLW008] [myindex][2] marking and sending shard failed due to [failed to update mapping for index]
org.elasticsearch.index.mapper.MapperParsingException: Failed to parse mapping [xyz]: Root mapping definition has unsupported parameters:  [_size : {enabled=true}]

[2020-04-12T10:58:32,370][WARN ][o.e.c.r.a.AllocationService] [CDDMLW008] failing shard [failed shard, shard [myindex][3], node[pmEaQ33FTnen2Hov2EDmuQ], [P], 
recovery_source[snapshot recovery [BB8lKfSLQt6lMaLfCzIyIg] from myrepo:snapshot_1/sGSyKr3VSf69wT0qx8gjGw], s[INITIALIZING], a[id=uMHmjwAsTDWinjWzyrPqYQ], 
unassigned_info[[reason=NEW_INDEX_RESTORED], at[2020-04-12T15:58:31.736Z], delayed=false, details[restore_source[myrepo/snapshot_1]], 
allocation_status[fetching_shard_data]], message [failed to update mapping for index], failure [MapperParsingException[Failed to parse mapping [xyz]: Root mapping 
definition has unsupported parameters:  [_size : {enabled=true}]]; nested: MapperParsingException[Root mapping definition has unsupported parameters:  [_size : {enabled=true}]]; ], 
markAsStale [true]]
org.elasticsearch.index.mapper.MapperParsingException: Failed to parse mapping [xyz]: Root mapping definition has unsupported parameters:  [_size : {enabled=true}]

Ok, Thank you for your throughout test.
We're close.

Could you stop the node, install this plugin and restart the node?

https://www.elastic.co/guide/en/elasticsearch/plugins/7.3/mapper-size.html

Thanks to @Luca_Belluccini for prompt response. Can you please provide pointer about steps to install the plugin in Windows? The above link allows me to download zip file, however the instructions are for Linux.

It's basically the same as Linux.

Go to your C:\Elasticsearch\elasticsearch-7.3.2\bin\ directory.

Then run elasticsearch-plugin.bat install mapper-size.

You will need internet access.

You will get something like this:

PS C:\elasticsearch-7.3.2\bin> .\elasticsearch-plugin.bat install mapper-size
-> Downloading mapper-size from elastic
[=================================================] 100%  
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.bouncycastle.jcajce.provider.drbg.DRBG (file:/C:/elasticsearch-7.3.2/lib/tools/plugin-cli/bcprov-jdk15on-1.61.jar) to constructor sun.security.provider.Sun()
WARNING: Please consider reporting this to the maintainers of org.bouncycastle.jcajce.provider.drbg.DRBG
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
-> Installed mapper-size

Thanks to @Luca_Belluccini for prompt support with this issue.
Glad to report that "restore" step works correctly after installation of mapper-size plugin.
I am able to retrieve from "myindex".
GET /myindex/_search
Status stays green now. GET _cat/health?v
I can go ahead and mark this as solution.
I also take the opportunity to thank @oneoneonepig for providing various troubleshooting tips.

GET /_cat/indices
green open .kibana_task_manager       swM7-6kSQxqTceT8rpA63w 1 0    2 4  57.4kb  57.4kb
green open bank                       dyXgSbtZTLqZGOWD07kgsQ 1 0 1000 0 427.9kb 427.9kb
green open myindex                    h2BR4uuqTVCZZ3hf2OFbhg 5 0  260 0   1.3mb   1.3mb
green open .kibana_1                  6VOkG87tTz2FwO_4sYfN0A 1 0    4 1  29.2kb  29.2kb

This case can be closed now.

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.