Issue help - snapshot creation failure

Elastic Stack Elasticsearch
21 
[2017-04-04 18:07:58,805][WARN ][transport                ] [master] Transport response handler not found of id [2424]
[2017-04-04 18:07:58,808][WARN ][transport                ] [master] Transport response handler not found of id [2425]
[2017-04-04 18:07:58,808][WARN ][rest.suppressed          ] path: /_snapshot/backup, params: {repository=backup}
RepositoryVerificationException[[backup] [PD8oAfGZQuiO_zyVxuduKw, 'RemoteTransportException[[dataNode2][10.85.7.54:9300][internal:admin/repository/verify]]; nested: RepositoryMissingException[[backup] missing];'], [wDj7ox5RSf6XiAhd4LEUgQ, 'RemoteTransportException[[dataNode1][10.85.7.53:9300][internal:admin/repository/verify]]; nested: RepositoryMissingException[[backup] missing];']]]
	at org.elasticsearch.repositories.RepositoriesService$VerifyingRegisterRepositoryListener$1.onResponse(RepositoriesService.java:440)
	at org.elasticsearch.repositories.RepositoriesService$VerifyingRegisterRepositoryListener$1.onResponse(RepositoriesService.java:436)
	at org.elasticsearch.repositories.RepositoriesService$3.onResponse(RepositoriesService.java:224)
	at org.elasticsearch.repositories.RepositoriesService$3.onResponse(RepositoriesService.java:214)
	at org.elasticsearch.repositories.VerifyNodeRepositoryAction.finishVerification(VerifyNodeRepositoryAction.java:116)
	at org.elasticsearch.repositories.VerifyNodeRepositoryAction$1.handleException(VerifyNodeRepositoryAction.java:107)
	at org.elasticsearch.transport.netty.MessageChannelHandler.handleException(MessageChannelHandler.java:184)
	at org.elasticsearch.transport.netty.MessageChannelHandler.handlerResponseError(MessageChannelHandler.java:174)
	at org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:122)
	at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
	at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
	at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
	at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
	at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
	at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
	at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
	at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
	at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
	at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
	at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
	at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
	at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
	at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
	at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
	at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
[2017-04-04 18:16:55,379][ERROR][license.plugin.core      ] [master] 
#
# LICENSE EXPIRED ON [Sunday, December 11, 2016]. IF YOU HAVE A NEW LICENSE, PLEASE
# UPDATE IT. OTHERWISE, PLEASE REACH OUT TO YOUR SUPPORT CONTACT.
# 
# COMMERCIAL PLUGINS OPERATING WITH REDUCED FUNCTIONALITY
# - marvel
#  - The agent will stop collecting cluster and indices metrics
#  - The agent will stop automatically cleaning indices older than [marvel.history.duration]
[2017-04-04 18:26:55,380][ERROR][license.plugin.core      ] [master]
22

I hope it won't get you too much trouble as I post the most recent logs and it was pretty big. I found there are some clues taking about the location which I have corrected to what you suggested but have no clue why they are still showing in that way. Let me know how you think about this.

Much appreciated.

23

Hi David,

I've posted the logs as you asked. Could you please take a look if
something can be figured out quickly?

Thanks a lot.

24

Apparently you tried to create a repository using /var/log/elasticsearch/backup dir but you whitelisted /mnt/elasticsearch in path.repo.

[2017-04-04 18:06:55,232][WARN ][repositories.fs          ] [master] The specified location [/var/log/elasticsearch/backup] doesn't start with any repository paths specified by the path.repo setting: [[/mnt/elasticsearch]]
25

I have no idea why it comes up in this way - I actually set the repository and restarted all services. Then it still comes up in this way. Should I keep bouncing the services until it's succeeded?
Thansk.

26

So may be you registered previously a repository and the master node is trying to instantiate it at boot time?

You can probably delete it first.

27

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.