# How to migrate older logs stored in one elasticsearch cluster to another new one?
**URL:** https://discuss.elastic.co/t/how-to-migrate-older-logs-stored-in-one-elasticsearch-cluster-to-another-new-one/360912
**Category:** Elasticsearch
**Tags:** elastic-stack-monitoring, ilm-index-lifecycle-management
**Created:** [June 6, 2024, 5:01am UTC](https://discuss.elastic.co/t/how-to-migrate-older-logs-stored-in-one-elasticsearch-cluster-to-another-new-one/360912 "2024-06-06T05:01:35Z")
**Posts on this page:** 11
**Page:** 1
### Author:  [@kriti\_dabas](https://discuss.elastic.co/u/kriti_dabas)
#### Post date: [June 6, 2024, 5:01am UTC](https://discuss.elastic.co/t/how-to-migrate-older-logs-stored-in-one-elasticsearch-cluster-to-another-new-one/360912/1 "2024-06-06T05:01:35Z")
I don't have any paid features of elasticsearch.
Older elasticsearch cluster version is 7.17 and the new one is 8.12.
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [June 6, 2024, 5:42am UTC](https://discuss.elastic.co/t/how-to-migrate-older-logs-stored-in-one-elasticsearch-cluster-to-another-new-one/360912/2 "2024-06-06T05:42:00Z")
Snapshot then restore.
---
### Author:  [@kriti\_dabas](https://discuss.elastic.co/u/kriti_dabas)
#### Post date: [June 7, 2024, 10:47am UTC](https://discuss.elastic.co/t/how-to-migrate-older-logs-stored-in-one-elasticsearch-cluster-to-another-new-one/360912/3 "2024-06-07T10:47:09Z")
just to be more precise i haven't upgraded from 7.17 to 8.12. These are two different clusters.
I want to move logs to a new cluster which is of version 8.12 and the older one is 7.17.
---
### Author:  [@kriti\_dabas](https://discuss.elastic.co/u/kriti_dabas)
#### Post date: [June 7, 2024, 10:50am UTC](https://discuss.elastic.co/t/how-to-migrate-older-logs-stored-in-one-elasticsearch-cluster-to-another-new-one/360912/4 "2024-06-07T10:50:52Z")
I have to go to snapshot and restore in stack management of older cluster and afterwards?
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [June 7, 2024, 11:20am UTC](https://discuss.elastic.co/t/how-to-migrate-older-logs-stored-in-one-elasticsearch-cluster-to-another-new-one/360912/5 "2024-06-07T11:20:21Z")
So
- upgrade to the latest 7.17
- go to the upgrade assistant and check that everything is ok
- if you are running on [cloud.elastic.co](http://cloud.elastic.co), just click the upgrade button
If not, follow the guide 😉 [Upgrade Elasticsearch | Elasticsearch Guide [8.14] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-upgrade.html)
---
### Author:  [@kriti\_dabas](https://discuss.elastic.co/u/kriti_dabas)
#### Post date: [June 11, 2024, 5:22am UTC](https://discuss.elastic.co/t/how-to-migrate-older-logs-stored-in-one-elasticsearch-cluster-to-another-new-one/360912/6 "2024-06-11T05:22:13Z")
without upgrading 7.17 cluster to 8.12 I cannot move logs from this cluster to the cluster version 8.12?
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [June 11, 2024, 12:08pm UTC](https://discuss.elastic.co/t/how-to-migrate-older-logs-stored-in-one-elasticsearch-cluster-to-another-new-one/360912/7 "2024-06-11T12:08:17Z")
Yes you can. With snapshot & restore which was my first answer.
- Snapshot in 7.x cluster
- Restore in 8.x cluster
---
### Author:  [@kriti\_dabas](https://discuss.elastic.co/u/kriti_dabas)
#### Post date: [June 12, 2024, 5:23am UTC](https://discuss.elastic.co/t/how-to-migrate-older-logs-stored-in-one-elasticsearch-cluster-to-another-new-one/360912/8 "2024-06-12T05:23:16Z")
what are the steps for taking snapshot from 7.17 version of elasticsearch and restoring logs in 8.12 version?
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [June 12, 2024, 9:31am UTC](https://discuss.elastic.co/t/how-to-migrate-older-logs-stored-in-one-elasticsearch-cluster-to-another-new-one/360912/9 "2024-06-12T09:31:25Z")
Have a look at [Snapshot and restore | Elasticsearch Guide [8.14] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/snapshot-restore.html)
Basically:
- [Create a repository](https://www.elastic.co/guide/en/elasticsearch/reference/current/snapshots-filesystem-repository.html) on the 7.X cluster
- [Create a Snapshot](https://www.elastic.co/guide/en/elasticsearch/reference/current/snapshots-take-snapshot.html)
- Share the FS folder on the new node or copy the files there over the network
- [Create a repository](https://www.elastic.co/guide/en/elasticsearch/reference/current/snapshots-filesystem-repository.html) on the 8.X cluster
- [Restore the snapshot](https://www.elastic.co/guide/en/elasticsearch/reference/current/snapshots-restore-snapshot.html)
If you are blocked at one of the steps, please share what is the issue or what you don't understand from the documentation. We'll be happy to help.
---
### Author:  [@kriti\_dabas](https://discuss.elastic.co/u/kriti_dabas)
#### Post date: [June 18, 2024, 9:23am UTC](https://discuss.elastic.co/t/how-to-migrate-older-logs-stored-in-one-elasticsearch-cluster-to-another-new-one/360912/10 "2024-06-18T09:23:14Z")
I made changes in all the yml files of elasticsearch nodes be it coordination, data and master.
path.repo: /mount/backups/my\_backup
but it shows so many failed shards in snapshot and restore on kibana.
There are many indices shown in my\_backup directory on one of the data node.
but the uuid doesn't match with any of the indices i wanted to take snapshot of as queried on kibana.
The UUID'S present their are different.
---
### Author:  [@Hadj\_Hassine\_Younes](https://discuss.elastic.co/u/Hadj_Hassine_Younes)
#### Post date: [December 9, 2024, 6:00pm UTC](https://discuss.elastic.co/t/how-to-migrate-older-logs-stored-in-one-elasticsearch-cluster-to-another-new-one/360912/11 "2024-12-09T18:00:14Z")
Hello @dadoonet ,
how are you?
i faced the same work , but i moved data from an ELK version 7.17.22 to an ECK-stack version 8.14.3 ( later I will upgrade to 8.16) managed by elastic-operator 2.14
I had severals problems :
1-
when i setted up a new APM ( eck-apm-server) , i found that the new eck-stack is managing apm indices with defferent way ( the documents structure seems changed ) so we cannot reindex ( remotly) from the legacy apm indices to the new one ( in eck-elasticsearch).
2-
the ilm policy, which seem weired to me .
for logstash indices ( they are feeded by fluentd) , i setted an ilm :
```auto
{
"policy": {
"phases": {
"hot": {
"actions": {
"rollover": {
"max_age": "7d",
"max_primary_shard_size": "5gb",
"max_docs": 15000000
},
"set_priority": {
"priority": 100
}
}
},
"warm": {
"min_age": "10d",
"actions": {
"set_priority": {
"priority": 50
},
"shrink": {
"number_of_shards": 1
},
"forcemerge": {
"max_num_segments": 1
}
}
},
"cold": {
"min_age": "30d",
"actions": {
"set_priority": {
"priority": 0
},
"allocate": {
"number_of_replicas": 0
}
}
},
"delete": {
"min_age": "90d",
"actions": {
"delete": {}
}
}
}
}
}
```
but when running
```auto
GET logstash-new-default-2024.12.04-000003/_ilm/explain
```
i got this output 🙂
```auto
{
"indices": {
"logstash-new-default-2024.12.04-000003": {
"index": "logstash-new-default-2024.12.04-000003",
"managed": true,
"policy": "custome-log-lifecycle",
"index_creation_date_millis": 1733350225293,
"time_since_index_creation": "4.78d",
"lifecycle_date_millis": 1733350225293,
"age": "4.78d",
"phase": "hot",
"phase_time_millis": 1733350225561,
"action": "rollover",
"action_time_millis": 1733350225761,
"step": "check-rollover-ready",
"step_time_millis": 1733350225761,
"phase_execution": {
"policy": "custome-log-lifecycle",
"phase_definition": {
"min_age": "0ms",
"actions": {
"rollover": {
"max_age": "7d",
"min_docs": 1,
"max_primary_shard_docs": 200000000,
"max_size": "15gb"
}
}
},
"version": 1,
"modified_date_in_millis": 1732800499186
}
}
}
}
```
as you can see here the rollover is not as i set in my ilm.
I deeper analyzed this and found that in the cluster settings, there's a default rollover configuration:
```json
"cluster.lifecycle.default.rollover": "max_age=auto,max_primary_shard_size=50gb,min_docs=1,max_primary_shard_docs=200000000"
```
According to the documentation ([Data stream lifecycle | Elasticsearch Guide [8.16] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/data-stream-lifecycle.html)), this setting is meant for data streams that don't have specific lifecycle settings, or for indices that aren't managed by ILM.
However, in my case:
1. These are regular indices (not data streams)
2. They're explicitly managed by ILM with a defined policy
3. They have proper index template configuration:
```auto
{
"name": "logstash-new",
"index_template": {
"index_patterns": ["logstash-new-*"],
"template": {
"settings": {
"index": {
"lifecycle": {
"name": "custome-log-lifecycle",
"rollover_alias": "logstash-new"
}
}
}
}
}
}
```
Despite this, the \_ilm/explain shows that the cluster default settings are being applied instead of my ILM policy settings. The indices aren't rolling over at my configured thresholds (15M docs, 5GB) but are using the much higher cluster defaults (200M docs, 15GB).
Is this the expected behavior? Should cluster.lifecycle.default.rollover settings override explicit ILM policy settings for ILM-managed indices? Or is this possibly a bug in the ECK operator or Elasticsearch 8.14.3?