# Preserving ML "Education" and Jobs **URL:** \ **Category:** Elasticsearch\ **Tags:** elastic-stack-machine-learning\ **Created:** [February 21, 2018, 11:36pm UTC](https://discuss.elastic.co/t/preserving-ml-education-and-jobs/120949 "2018-02-21T23:36:44Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![michaelm14](https://avatars.discourse-cdn.com/v4/letter/m/13edae/32.png) [@michaelm14](https://discuss.elastic.co/u/michaelm14)\ **Post date:** [February 21, 2018, 11:36pm UTC](https://discuss.elastic.co/t/preserving-ml-education-and-jobs/120949/1 "2018-02-21T23:36:44Z")
We have multiple clusters running autonomously and we've created ML jobs within a Dev environment. I am admittedly a real newbie with ML and AI, but my understanding is the algorithms are continuously updating based on analyzed data for better accuracy. I see the blue areas showing upper and lower expectations, so I believe it is learning more as time progresses, correct? If that's the case, how can I migrate my "experienced" algorithms from the Dev environment to a Production one without restarting the training? In other words, where do the algorithms live? Along with that, I would hope the process would allow me to backup/archive existing, "educated" jobs so I don't have to start over in the event of a rebuild. The JSON for existing jobs has parameters and configuration info, but I don't see any algorithm information and certainly no history of what it has already learned. I know I'm really new at ML so if anyone has suggestions, please let me know. Thank you! ---
**Author:** ![dmitri](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dmitri/32/17997_2.png) [@dmitri](https://discuss.elastic.co/u/dmitri)\ **Post date:** [February 23, 2018, 1:51pm UTC](https://discuss.elastic.co/t/preserving-ml-education-and-jobs/120949/2 "2018-02-23T13:51:05Z")
Hi Michael, > I see the blue areas showing upper and lower expectations, so I believe it is learning more as time progresses, correct? Correct. The model learns online thus it keeps changing as data comes in and time moves forward. > If that's the case, how can I migrate my "experienced" algorithms from the Dev environment to a Production one without restarting the training? In other words, where do the algorithms live? ML jobs store their state in an index in the cluster. Unfortunately, if you want to migrate your jobs to a different cluster, it is not possible. You will have to start over. Hopefully, it won't take long to process historic data and catch up with real-time analysis. > Along with that, I would hope the process would allow me to backup/archive existing, "educated" jobs so I don't have to start over in the event of a rebuild. The JSON for existing jobs has parameters and configuration info, but I don't see any algorithm information and certainly no history of what it has already learned. You will notice that a job may have a `model_snapshot_id`. This links the job to its current model state. We have a set of APIs that allow you to manage past model snapshots including the ability to revert to a previous one. Model snapshots are taken periodically, based on the job config parameter `background_persist_interval`. Also, the parameters `model_snapshot_retention_days` dictates how old should a snapshot be before it is automatically removed. You can read more about job config options in [here](https://www.elastic.co/guide/en/elasticsearch/reference/6.2/ml-put-job.html#_request_body_6). You can read more about the model snapshot management APIs [here](https://www.elastic.co/guide/en/x-pack/6.2/ml-api-quickref.html#ml-api-snapshots). ---
**Author:** ![michaelm14](https://avatars.discourse-cdn.com/v4/letter/m/13edae/32.png) [@michaelm14](https://discuss.elastic.co/u/michaelm14)\ **Post date:** [February 23, 2018, 2:27pm UTC](https://discuss.elastic.co/t/preserving-ml-education-and-jobs/120949/3 "2018-02-23T14:27:52Z")
dmitri, Thank you very much for detailed answer! That helps a lot. I'll work on tuning the model snapshots to help with the backup strategy. Thanks again! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [March 23, 2018, 2:28pm UTC](https://discuss.elastic.co/t/preserving-ml-education-and-jobs/120949/4 "2018-03-23T14:28:06Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.