Google Cloud Storage Repository Plugin - "state" : "PARTIAL"

Hello,

I'm using Elastic (6.2.2) Stack w/ Google Cloud Storage Repository Plugin | Elasticsearch Plugins and Integrations.

I keep getting "PARTIAL" instead of "SUCCESS" state of my snapshot:

# time curl --silent --header 'Content-Type: application/json' --request PUT "$ELASTICSEARCH_URI/_snapshot/repository-gcs/X?wait_for_completion=true&pretty" --data '{"indices":"X"}'
{
  "snapshot" : {
    "snapshot" : "X",
    "uuid" : "NuP6yLBYQoq0ZsChRBS04g",
    "version_id" : 6020299,
    "version" : "6.2.2",
    "indices" : [
      "X"
    ],
    "include_global_state" : true,
    "state" : "PARTIAL",
    "start_time" : "2018-04-08T19:02:59.717Z",
    "start_time_in_millis" : 1523214179717,
    "end_time" : "2018-04-08T19:03:01.184Z",
    "end_time_in_millis" : 1523214181184,
    "duration_in_millis" : 1467,
    "failures" : [
      {
        "index" : "X",
        "index_uuid" : "X",
        "shard_id" : 3,
        "reason" : "RepositoryMissingException[[repository-gcs] missing]",
        "node_id" : "LPVtuXogTkiB7sk8ibnI0w",
        "status" : "INTERNAL_SERVER_ERROR"
      }
    ],
    "shards" : {
      "total" : 5,
      "failed" : 1,
      "successful" : 4
    }
  }
}

real	0m3.194s
user	0m0.004s
sys	0m0.004s
# 

elasticsearch log (master):

esm1             | [2018-04-08T19:03:00,391][INFO ][o.e.s.SnapshotsService   ] [esm1] snapshot [repository-gcs:X/NuP6yLBYQoq0ZsChRBS04g] started
esm1             | [2018-04-08T19:03:02,606][INFO ][o.e.s.SnapshotsService   ] [esm1] snapshot [repository-gcs:X/NuP6yLBYQoq0ZsChRBS04g] completed with state [PARTIAL]

elasticsearch log (data):

esd2             | [2018-04-08T19:03:00,413][WARN ][o.e.s.SnapshotShardsService] [esd2] [[X][3]][repository-gcs:X/NuP6yLBYQoq0ZsChRBS04g] failed to snapshot shard
esd2             | org.elasticsearch.repositories.RepositoryMissingException: [repository-gcs] missing
esd2             | 	at org.elasticsearch.repositories.RepositoriesService.repository(RepositoriesService.java:335) ~[elasticsearch-6.2.2.jar:6.2.2]
esd2             | 	at org.elasticsearch.snapshots.SnapshotShardsService.snapshot(SnapshotShardsService.java:408) ~[elasticsearch-6.2.2.jar:6.2.2]
esd2             | 	at org.elasticsearch.snapshots.SnapshotShardsService.access$200(SnapshotShardsService.java:98) ~[elasticsearch-6.2.2.jar:6.2.2]
esd2             | 	at org.elasticsearch.snapshots.SnapshotShardsService$1.doRun(SnapshotShardsService.java:355) [elasticsearch-6.2.2.jar:6.2.2]
esd2             | 	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:672) [elasticsearch-6.2.2.jar:6.2.2]
esd2             | 	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.2.2.jar:6.2.2]
esd2             | 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_161]
esd2             | 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_161]
esd2             | 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_161]

it works for some, however not for all(

Please advise.

What is the output of

GET /_cat/plugins?v
# curl --silent --request GET $ELASTICSEARCH_URI/_cat/plugins?v
name             component          version
esc3             ingest-geoip       6.2.2
esc3             ingest-user-agent  6.2.2
esc3             repository-gcs     6.2.2
esc3             x-pack-core        6.2.2
esc3             x-pack-deprecation 6.2.2
esc3             x-pack-graph       6.2.2
esc3             x-pack-logstash    6.2.2
esc3             x-pack-ml          6.2.2
esc3             x-pack-monitoring  6.2.2
esc3             x-pack-security    6.2.2
esc3             x-pack-upgrade     6.2.2
esc3             x-pack-watcher     6.2.2
esd2             ingest-geoip       6.2.2
esd2             ingest-user-agent  6.2.2
esd2             repository-gcs     6.2.2
esd2             x-pack-core        6.2.2
esd2             x-pack-deprecation 6.2.2
esd2             x-pack-graph       6.2.2
esd2             x-pack-logstash    6.2.2
esd2             x-pack-ml          6.2.2
esd2             x-pack-monitoring  6.2.2
esd2             x-pack-security    6.2.2
esd2             x-pack-upgrade     6.2.2
esd2             x-pack-watcher     6.2.2
esm1             ingest-geoip       6.2.2
esm1             ingest-user-agent  6.2.2
esm1             repository-gcs     6.2.2
esm1             x-pack-core        6.2.2
esm1             x-pack-deprecation 6.2.2
esm1             x-pack-graph       6.2.2
esm1             x-pack-logstash    6.2.2
esm1             x-pack-ml          6.2.2
esm1             x-pack-monitoring  6.2.2
esm1             x-pack-security    6.2.2
esm1             x-pack-upgrade     6.2.2
esm1             x-pack-watcher     6.2.2
esd5             ingest-geoip       6.2.2
esd5             ingest-user-agent  6.2.2
esd5             repository-gcs     6.2.2
esd5             x-pack-core        6.2.2
esd5             x-pack-deprecation 6.2.2
esd5             x-pack-graph       6.2.2
esd5             x-pack-logstash    6.2.2
esd5             x-pack-ml          6.2.2
esd5             x-pack-monitoring  6.2.2
esd5             x-pack-security    6.2.2
esd5             x-pack-upgrade     6.2.2
esd5             x-pack-watcher     6.2.2
esm3             ingest-geoip       6.2.2
esm3             ingest-user-agent  6.2.2
esm3             repository-gcs     6.2.2
esm3             x-pack-core        6.2.2
esm3             x-pack-deprecation 6.2.2
esm3             x-pack-graph       6.2.2
esm3             x-pack-logstash    6.2.2
esm3             x-pack-ml          6.2.2
esm3             x-pack-monitoring  6.2.2
esm3             x-pack-security    6.2.2
esm3             x-pack-upgrade     6.2.2
esm3             x-pack-watcher     6.2.2
esc1             ingest-geoip       6.2.2
esc1             ingest-user-agent  6.2.2
esc1             repository-gcs     6.2.2
esc1             x-pack-core        6.2.2
esc1             x-pack-deprecation 6.2.2
esc1             x-pack-graph       6.2.2
esc1             x-pack-logstash    6.2.2
esc1             x-pack-ml          6.2.2
esc1             x-pack-monitoring  6.2.2
esc1             x-pack-security    6.2.2
esc1             x-pack-upgrade     6.2.2
esc1             x-pack-watcher     6.2.2
esd1             ingest-geoip       6.2.2
esd1             ingest-user-agent  6.2.2
esd1             repository-gcs     6.2.2
esd1             x-pack-core        6.2.2
esd1             x-pack-deprecation 6.2.2
esd1             x-pack-graph       6.2.2
esd1             x-pack-logstash    6.2.2
esd1             x-pack-ml          6.2.2
esd1             x-pack-monitoring  6.2.2
esd1             x-pack-security    6.2.2
esd1             x-pack-upgrade     6.2.2
esd1             x-pack-watcher     6.2.2
elasticsearch ingest-geoip       6.2.2
elasticsearch ingest-user-agent  6.2.2
elasticsearch repository-gcs     6.2.2
elasticsearch x-pack-core        6.2.2
elasticsearch x-pack-deprecation 6.2.2
elasticsearch x-pack-graph       6.2.2
elasticsearch x-pack-logstash    6.2.2
elasticsearch x-pack-ml          6.2.2
elasticsearch x-pack-monitoring  6.2.2
elasticsearch x-pack-security    6.2.2
elasticsearch x-pack-upgrade     6.2.2
elasticsearch x-pack-watcher     6.2.2
esd4             ingest-geoip       6.2.2
esd4             ingest-user-agent  6.2.2
esd4             repository-gcs     6.2.2
esd4             x-pack-core        6.2.2
esd4             x-pack-deprecation 6.2.2
esd4             x-pack-graph       6.2.2
esd4             x-pack-logstash    6.2.2
esd4             x-pack-ml          6.2.2
esd4             x-pack-monitoring  6.2.2
esd4             x-pack-security    6.2.2
esd4             x-pack-upgrade     6.2.2
esd4             x-pack-watcher     6.2.2
esd3             ingest-geoip       6.2.2
esd3             ingest-user-agent  6.2.2
esd3             repository-gcs     6.2.2
esd3             x-pack-core        6.2.2
esd3             x-pack-deprecation 6.2.2
esd3             x-pack-graph       6.2.2
esd3             x-pack-logstash    6.2.2
esd3             x-pack-ml          6.2.2
esd3             x-pack-monitoring  6.2.2
esd3             x-pack-security    6.2.2
esd3             x-pack-upgrade     6.2.2
esd3             x-pack-watcher     6.2.2
esm2             ingest-geoip       6.2.2
esm2             ingest-user-agent  6.2.2
esm2             repository-gcs     6.2.2
esm2             x-pack-core        6.2.2
esm2             x-pack-deprecation 6.2.2
esm2             x-pack-graph       6.2.2
esm2             x-pack-logstash    6.2.2
esm2             x-pack-ml          6.2.2
esm2             x-pack-monitoring  6.2.2
esm2             x-pack-security    6.2.2
esm2             x-pack-upgrade     6.2.2
esm2             x-pack-watcher     6.2.2
#

Is there anything else in logs (in all logs)?

I tail all nodes logs, then execute curl command (logs that I included in first post is what happens after). Is there anything else I can provide to help you troubleshoot?

I'm running out of ideas. May be @Igor_Motov or @tanguy have an idea.
Looks like one node is not aware of the repository which I would only explain with a split brain and a cluster state not correctly sent to all nodes but I think that in that case logs would tell that.

Could you remove the repository? Create it again?

@alexus do all nodes have the same settings (including secure settings)? Did you restart nodes after updating these settings?

@Igor_Motov Apart for node.* settings (as I have dedicated master, data, ingest and coordinate node), as well memory settings, all nodes are exactly identical.

Which security settings or changes are you referring too?

Per Repository Verification:

When a repository is registered, it’s immediately verified on all master and data nodes to make sure that it is functional on all nodes currently present in the cluster. The verify parameter can be used to explicitly disable the repository verification when registering or updating a repository:

... needless to say I did not use ?verify=false when registered my repository.

However after deleting my repository and re-registering it again, followed by creating new snapshot with new state SUCCESS!

Thanks!

I was referring to GCS settings. The problem that you describe typically occur when node settings on nodes are getting out of sync or a plugin is not installed on some nodes during the repository registration.

@Igor_Motov I see, in my case I use Dockerfile (where I integrate GCS settings). I don't see how or why would one of the nodes, somehow get out of sync or loose plugin that was installed into image, but it looks like somehow it did get out of sync as per David's requirements I deleted and re-registered repository and it seems to address my issue.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.