Restoring machine learning config from a snapshot

Elastic Stack Elasticsearch
1

I am migrating my on-premise ELK stack to elastic cloud as per this article

I use the snapshot method and I did restored all the .ml* indices Unfortunately, I cannot retreive my ML job configurations. According to the doc, the config is in .ml-config index, which I cannot see when requesting:

GET /_cat/indices/.ml*

green open .ml-state            nlE3yctYSYSHJgcN_SEuKQ 5 1   2943 31 513.5mb 256.7mb
green open .ml-anomalies-shared bpvA1bNnR_iWsKtrocb5dQ 5 1 354785 14 148.1mb    74mb
green open .ml-notifications    Flwftz4USw2MZApxzG9rAA 1 1    965  0 412.5kb 195.8kb

How can I retreive my ML configuration?

regards

2

The .ml-config indices are new to v6.6. Prior to that, ML job configs were stored in cluster state.

The best option is to export the job and datafeed config's via the API:

_xpack/ml/anomaly_detectors/${JOB_ID}?pretty
_xpack/ml/datafeeds/datafeed-${JOB_ID}?pretty

Then, refactor and "PUT" those configs on the new cluster. From there you'd need to rebuild the models and results on the raw data on the new cluster.

In the future, we'll have a better facility to move everything from one system to another. Moving the job configs out of cluster state to .ml-config was the first step!

3

It means that a snapshot is not enough to recover? that is the same for the users and roles, right?

Especially, the article to migrate to the cloud is not enough? can you give me the whole process, then?

4

The article referenced above (migrating to cloud) was written in 2016, before ML was released - it therefore does not contain information on how to migrate ML configurations to another system.

Let me re-iterate - if you want to move ML job configurations from one system to another, you need to follow the process I mentioned above - you need to extract all existing configs with the API and you need to PUT those configs on the new cluster. From there you'd need to rebuild the models and results of the ML jobs from the raw data on the new cluster.

5

OK, that's clear. what about the users and roles?

6

Pretty sure those are stored in the .security index - so probably can re-index from remote to move those. But, I'm not an expert in that part of the product. Maybe head on over to https://discuss.elastic.co/tags/c/elasticsearch/security

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.