# Moving ML jobs and their data

**URL:** <https://discuss.elastic.co/t/moving-ml-jobs-and-their-data/136635>\
**Category:** Elasticsearch\
**Tags:** elastic-stack-machine-learning\
**Created:** [June 20, 2018, 8:13am UTC](https://discuss.elastic.co/t/moving-ml-jobs-and-their-data/136635 "2018-06-20T08:13:37Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![blueren](https://avatars.discourse-cdn.com/v4/letter/b/ecae2f/32.png) [@blueren](https://discuss.elastic.co/u/blueren)\
**Post date:** [June 20, 2018, 8:13am UTC](https://discuss.elastic.co/t/moving-ml-jobs-and-their-data/136635/1 "2018-06-20T08:13:37Z")

</div>

I have a a few ml jobs saved and running in my local build. I would like to somehow export the same jobs on to another build that is running in a different machine. Is this possible? If so, how?

Also, assuming that I've moved my indexes also to the new build, is there a possibility of moving the ml related data (the anomalies and models) Also onto the new build?

Thanks.

---

<div class="post-metadata">

**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\
**Post date:** [June 20, 2018, 10:40am UTC](https://discuss.elastic.co/t/moving-ml-jobs-and-their-data/136635/2 "2018-06-20T10:40:14Z")

</div>

Hello,

There currently is no officially supported way to export ML job configurations, models, and results as a unit and move to a new system. There are documented enhancement requests to add this capability in the future.

The current recommended procedure is to

1. manually, or programmatically copy ML job configurations from the old system to the new system
2. re-train the ML models using historical data on the new system
3. allow the new job(s) to run on-going on real-time data coming into the new system

---

<div class="post-metadata">

**Author:** ![blueren](https://avatars.discourse-cdn.com/v4/letter/b/ecae2f/32.png) [@blueren](https://discuss.elastic.co/u/blueren)\
**Post date:** [June 20, 2018, 10:45am UTC](https://discuss.elastic.co/t/moving-ml-jobs-and-their-data/136635/3 "2018-06-20T10:45:14Z")

</div>

Thanks. I've moved some of my indices from the old system on to the new one using this tool called [elatsticdump](https://www.npmjs.com/package/elasticdump). I'm not sure what Indeces refer to the ml data.  
I see the below indices. Do I copy them over as well?

```
.ml-anomalies
.ml-notifications

```

I've been lead to believe that both the training rules as well as the model is being stored in ES. Where exactly can I find them?

On another note, if I were to retrain the model, will it be identical to what was existing in the old machine? Or will there be a variance?

---

<div class="post-metadata">

**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\
**Post date:** [June 20, 2018, 10:53am UTC](https://discuss.elastic.co/t/moving-ml-jobs-and-their-data/136635/4 "2018-06-20T10:53:43Z")

</div>

- `.ml-anomalies-*` are the anomaly result documents (the output of analysis)
- `.ml-notifications` are the "Job messages" seen on the Job Management page of the UI
- `.ml-state` is the serialized version of the job's statistical models
- `.ml-meta` contains meta information like Special Calendar days (introduced in v6.2)
- The job configurations themselves are stored in the cluster state.

Bottom line, it is not easy to just move everything over, hence my earlier comment about there not being an officially supported mechanism.

---

<div class="post-metadata">

**Author:** ![blueren](https://avatars.discourse-cdn.com/v4/letter/b/ecae2f/32.png) [@blueren](https://discuss.elastic.co/u/blueren)\
**Post date:** [June 20, 2018, 10:58am UTC](https://discuss.elastic.co/t/moving-ml-jobs-and-their-data/136635/5 "2018-06-20T10:58:01Z")

</div>

Thanks for the quick info @richcollier!

---

<div class="post-metadata">

**Author:** ![blueren](https://avatars.discourse-cdn.com/v4/letter/b/ecae2f/32.png) [@blueren](https://discuss.elastic.co/u/blueren)\
**Post date:** [June 20, 2018, 11:02am UTC](https://discuss.elastic.co/t/moving-ml-jobs-and-their-data/136635/6 "2018-06-20T11:02:33Z")

</div>

On the same subject, since I've been running the entire stack as separate docker containers, I could also just tgz the entire elastic data directory and move it over to the new system. This has worked but added to the actual data, it also has the licensing data - something which I do now want moved over. Are there specific indices for licensing as well? So that I may have them deleted prior to movement? I am currently running on trial licence.

---

<div class="post-metadata">

**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\
**Post date:** [June 20, 2018, 12:41pm UTC](https://discuss.elastic.co/t/moving-ml-jobs-and-their-data/136635/7 "2018-06-20T12:41:22Z")

</div>

License information is stored in the cluster state (and is persisted to disk to `path.data` in the `_state` directory).

Safest way to manage licenses is with the API - [https://www.elastic.co/guide/en/elasticsearch/reference/6.x/licensing-apis.html](https://www.elastic.co/guide/en/elasticsearch/reference/6.x/licensing-apis.html)

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [October 30, 2018, 10:21pm UTC](https://discuss.elastic.co/t/moving-ml-jobs-and-their-data/136635/8 "2018-10-30T22:21:36Z")

</div>



---

<div class="post-metadata">

**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\
**Post date:** [October 13, 2021, 11:38am UTC](https://discuss.elastic.co/t/moving-ml-jobs-and-their-data/136635/9 "2021-10-13T11:38:55Z")

</div>

[Now possible in v7.15](https://github.com/elastic/kibana/pull/101037)
