I need to change the location of the repository:
but when I do I get this error:
we had to move the mountpoint for the shared disk...
Hm that looks like a bug, but without a stack trace it's going to be hard to understand/fix it. Can you try and do the same thing directly with the create snapshot repo API rather than through the UI (i.e. using the dev console) and add the ?error_trace parameter to the URL to get a full stack trace?
PUT /_snapshot/daily?error_trace
{
"type": "fs",
"settings": {
"location": "/home/elasticsearch/esbackups/"
}
}
{
"error" : {
"root_cause" : [
{
"type" : "null_pointer_exception",
"reason" : "Cannot invoke \"org.elasticsearch.repositories.Repository.getMetadata()\" because \"repository\" is null",
"stack_trace" : "[Cannot invoke \"org.elasticsearch.repositories.Repository.getMetadata()\" because \"repository\" is null]; nested: NullPointerException[Cannot invoke \"org.elasticsearch.repositories.Repository.getMetadata()\" because \"repository\" is null];\n\tat org.elasticsearch.ElasticsearchException.guessRootCauses(ElasticsearchException.java:667)\n\tat org.elasticsearch.ElasticsearchException.generateFailureXContent(ElasticsearchException.java:595)\n\tat org.elasticsearch.rest.BytesRestResponse.build(BytesRestResponse.java:140)\n\tat org.elasticsearch.rest.BytesRestResponse.<init>(BytesRestResponse.java:101)\n\tat org.elasticsearch.rest.BytesRestResponse.<init>(BytesRestResponse.java:81)\n\tat org.elasticsearch.rest.action.RestActionListener.onFailure(RestActionListener.java:55)\n\tat org.elasticsearch.action.support.TransportAction$1.onFailure(TransportAction.java:97)\n\tat org.elasticsearch.action.support.ContextPreservingActionListener.onFailure(ContextPreservingActionListener.java:38)\n\tat org.elasticsearch.action.ActionListener$Delegating.onFailure(ActionListener.java:66)\n\tat org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction$1.handleException(TransportMasterNodeAction.java:248)\n\tat org.elasticsearch.transport.TransportService$4.handleException(TransportService.java:853)\n\tat org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1481)\n\tat org.elasticsearch.transport.InboundHandler.lambda$handleException$3(InboundHandler.java:380)\n\tat org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:291)\n\tat org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:378)\n\tat org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:370)\n\tat org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:134)\n\tat org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:88)\n\tat org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:743)\n\tat org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:147)\n\tat org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:119)\n\tat org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:84)\n\tat org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:71)\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)\n\tat io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)\n\tat io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:280)\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)\n\tat io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)\n\tat io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)\n\tat io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)\n\tat io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1383)\n\tat io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1246)\n\tat io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1295)\n\tat io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:529)\n\tat io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:468)\n\tat io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290)\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)\n\tat io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)\n\tat io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)\n\tat io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)\n\tat io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)\n\tat io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)\n\tat io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:689)\n\tat io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:652)\n\tat io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)\n\tat io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)\n\tat io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)\n\tat java.base/java.lang.Thread.run(Thread.java:1623)\nCaused by: java.lang.NullPointerException: Cannot invoke \"org.elasticsearch.repositories.Repository.getMetadata()\" because \"repository\" is null\n\tat org.elasticsearch.repositories.RepositoriesService.canUpdateInPlace(RepositoriesService.java:524)\n\tat org.elasticsearch.repositories.RepositoriesService.access$200(RepositoriesService.java:69)\n\tat org.elasticsearch.repositories.RepositoriesService$1.execute(RepositoriesService.java:211)\n\tat org.elasticsearch.cluster.ClusterStateUpdateTask.execute(ClusterStateUpdateTask.java:51)\n\tat org.elasticsearch.cluster.service.MasterService.executeTasks(MasterService.java:840)\n\tat org.elasticsearch.cluster.service.MasterService.calculateTaskOutputs(MasterService.java:407)\n\tat org.elasticsearch.cluster.service.MasterService.runTasks(MasterService.java:243)\n\tat org.elasticsearch.cluster.service.MasterService.access$100(MasterService.java:63)\n\tat org.elasticsearch.cluster.service.MasterService$Batcher.run(MasterService.java:170)\n\tat org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:146)\n\tat org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:202)\n\tat org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:718)\n\tat org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:262)\n\tat org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:225)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.lang.Thread.run(Thread.java:1623)\n"
}
],
"type" : "null_pointer_exception",
"reason" : "Cannot invoke \"org.elasticsearch.repositories.Repository.getMetadata()\" because \"repository\" is null",
"stack_trace" : "java.lang.NullPointerException: Cannot invoke \"org.elasticsearch.repositories.Repository.getMetadata()\" because \"repository\" is null\n\tat org.elasticsearch.repositories.RepositoriesService.canUpdateInPlace(RepositoriesService.java:524)\n\tat org.elasticsearch.repositories.RepositoriesService.access$200(RepositoriesService.java:69)\n\tat org.elasticsearch.repositories.RepositoriesService$1.execute(RepositoriesService.java:211)\n\tat org.elasticsearch.cluster.ClusterStateUpdateTask.execute(ClusterStateUpdateTask.java:51)\n\tat org.elasticsearch.cluster.service.MasterService.executeTasks(MasterService.java:840)\n\tat org.elasticsearch.cluster.service.MasterService.calculateTaskOutputs(MasterService.java:407)\n\tat org.elasticsearch.cluster.service.MasterService.runTasks(MasterService.java:243)\n\tat org.elasticsearch.cluster.service.MasterService.access$100(MasterService.java:63)\n\tat org.elasticsearch.cluster.service.MasterService$Batcher.run(MasterService.java:170)\n\tat org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:146)\n\tat org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:202)\n\tat org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:718)\n\tat org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:262)\n\tat org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:225)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.lang.Thread.run(Thread.java:1623)\n"
},
"status" : 500
}
now to work around the "bug" ...
If I delete the repository in kibana I assume it will remove the snapshots, which we definitely don't want. If I rename the directory on disk and create a empty one in its place, then do a create repository with the new location should that work?
Removing a repository does not remove the snapshots.
But you don't need to remove the repository. You can just create a new one with a new name.
I deleted the repository and recreated it with the correct path. That worked. I then repeated that for all the other repositories.
I realised when I tried to change the path the new path I gave was to the directory where the repository was to be created. I had assumed the the directory would be created with the same name as the repository. Not so.
Before deleting are recreating the repositories I tried "edit" and changing the path, this time using the full path and that still failed???
What version are you using exactly @Russell_Fulton?
I opened NPE when updating repository settings · Issue #100183 · elastic/elasticsearch · GitHub to investigate this further.
Ah I found a matching issue, NPE in RepositoriesService · Issue #85550 · elastic/elasticsearch · GitHub, fixed in 8.3.0 but not backported to 7.17 it seems.
2 Likes
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.