Hello Team,
I am currently testing snapshot and restore process in elasticsearch. After restoring a source only snapshot from s3 , it is not showing any doc counts and storage size. do we need to do any thing here to start searching this indices for data. Please help
As detailed in the documentation you will need to reindex the data after you have restored the snapshot.
Thanks Christian, i tried to reindex one of the index but it not creating a new index after reindex .
POST _reindex
{
"source": {
"index": "auditbeat-7.6.2-2020.07.22-000036"
},
"dest": {
"index": "auditbeat-7.6.2-2020.07.22-restored",
"version_type": "external"
}
}
{
"took" : 0,
"timed_out" : false,
"total" : 0,
"updated" : 0,
"created" : 0,
"deleted" : 0,
"batches" : 0,
"version_conflicts" : 0,
"noops" : 0,
"retries" : {
"bulk" : 0,
"search" : 0
},
"throttled_millis" : 0,
"requests_per_second" : -1.0,
"throttled_until_millis" : 0,
"failures" :
}
Please help
@Christian_Dahlqvist - What does this part of the doc mean, i.e. how would you get the right mappints, or will the reindex pick this up automatically?
" The mapping of the restored index is empty, but the original mapping is available from the types top level meta element."
Am not sure what is wrong. Will need to leave that for someone else.
@jijo.john- Any stats for "auditbeat-7.6.2-2020.07.22-000036" ? Or it's status maybe not green or recovered, etc?
From /auditbeat-7.6.2-2020.07.22-000036/_stats
Also /auditbeat-7.6.2-2020.07.22-000036/_settings
Have you tried without the version_type external? Seems not relevant if no existing docs in the index.
What were the mappings of the original index?
Hi Steve,
The index status is red , i restored it with 0 replica.
Please see the output below
{
"_shards" : {
"total" : 1,
"successful" : 0,
"failed" : 0
},
"_all" : {
"primaries" : { },
"total" : { }
},
"indices" : { }
}
{
"index-7.6.2-2020.07.22-000036" : {
"settings" : {
"index" : {
"mapping" : {
"total_fields" : {
"limit" : "10000"
}
},
"source_only" : "true",
"refresh_interval" : "5s",
"blocks" : {
"write" : "true"
},
"provided_name" : "<auditbeat-7.6.2-{now/d}-000036>",
"query" : {
"default_field" : [
"message",
"tags",
"agent.ephemeral_id",
"agent.id",
"agent.name",
"system.audit.user.password.type",
"fields.*"
]
},
"frozen" : "true",
"creation_date" : "1595391714148",
"number_of_replicas" : "0",
"uuid" : "0HWx4QMaSs6U5HSBVupQ9g",
"version" : {
"created" : "7060299"
},
"lifecycle" : {
"name" : "auditbeat",
"rollover_alias" : "auditbeat-7.6.2",
"indexing_complete" : "true"
},
"codec" : "best_compression",
"routing" : {
"allocation" : {
"require" : {
"box_type" : "warm"
}
}
},
"search" : {
"throttled" : "true"
},
"number_of_shards" : "1"
}
}
}
}
Okay but red is likely why this not work - it has no primary (replica = 0 has no effect and should make it green, as won't be looking for replicas).
You have allocation routing, too - for box warm - is that able to be satisfied here? Or else maybe you have no shards as they can't find a node that's warm.
Suggest get shard list/status for this index and an allocation explain on that shard to understand issue; might be obvious, but you have to fix this before it has any data to reindex, I think.
GET /_cat/shards?v&h=n,index,shard,prirep,state,sto,sc,unassigned.reason,unassigned.details&s=sto,index
GET /_cluster/allocation/explain
{"index": "auditbeat-7.6.2-2020.07.22-000036", "shard": 0,
"primary": true}
Why is this index frozen ? Is that part of source restore?
From what I read, the restored index is functional in that you can query the source data, etc. so I'd think should not be red, just read-only limited query; those aren't the same thing so something seems wrong but maybe someone who knows more about restores has info.
Please see the Output below
n index shard prirep state sto sc unassigned.reason unassigned.details
index-7.6.2-2020.07.22-000036 0 p UNASSIGNED INDEX_REOPENED
index-7.6.2-2020.07.22-000036 0 r UNASSIGNED INDEX_REOPENED
index-7.6.2-2020.07.23-000037 0 p UNASSIGNED INDEX_REOPENED
index-7.6.2-2020.07.23-000037 0 r UNASSIGNED INDEX_REOPENED
index-7.6.2-2020.07.27-000039 0 p UNASSIGNED INDEX_REOPENED
index-7.6.2-2020.07.27-000039 0 r UNASSIGNED INDEX_REOPENED
{
"index" : "index-7.6.2-2020.07.23-000037",
"shard" : 0,
"primary" : true,
"current_state" : "unassigned",
"unassigned_info" : {
"reason" : "INDEX_REOPENED",
"at" : "2020-07-30T13:42:14.606Z",
"last_allocation_status" : "no_valid_shard_copy"
},
"can_allocate" : "no_valid_shard_copy",
"allocate_explanation" : "cannot allocate because a previous copy of the primary shard existed but can no longer be found on the nodes in the cluster",
"node_allocation_decisions" : [
{
"node_id" : "6Kc_4PJcTyKUPeAzvKRUCQ",
"node_name" : "ibs-dc-siem-1",
"transport_address" : "10.186.10.176:9300",
"node_attributes" : {
"ml.machine_memory" : "33170718720",
"xpack.installed" : "true",
"ml.max_open_jobs" : "20"
},
"node_decision" : "no",
"store" : {
"found" : false
}
},
{
"node_id" : "O25dSwZSRaec8-cQh5Mfpg",
"node_name" : "ibs-dc-siem-backup",
"transport_address" : "10.186.10.12:9300",
"node_attributes" : {
"xpack.installed" : "true",
"box_type" : "warm"
},
"node_decision" : "no",
"store" : {
"found" : false
}
}
]
}
Index Freezing we enabled it as a part of roll over
Well, this all seems weird - this index was restored, but has no shards? And any shard it had was lost, how is that possible? And it's frozen for rollover, even though it's a restored index you are working ?
All strange - I guess I'd try to restore it again and immediately see its status, shards, etc. Long time since I restored, so I'm not sure what to expect, but docs talk about how it's queryable for _source and read-only, so I'd expect it to be green, just in limited duty until you reindex it.
Thanks Steve for your support , i was away for few days , Let me delete all the backups and take a fresh backup for testing
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.
