Machine Learning Functionality Across Clusters

Hello Elastic Community!

Running into an issue in my environment that I was hoping y'all could help with. My current architecture is divided into 3 clusters: RED1, BLUE1, and my Cross Cluster Search (CCS) cluster that has my CCS node and my Kibana node running on it. I have a Machine Learning node on RED1, but I can't get my CCS cluster, and by extension my Kibana, to read that there is an ML node out on RED1.

NOTE: Everything else is flowing between these clusters, only thing that is hung up is this ML thing. Looking at it from the Kibana in RED1 I can utilize ML, but the "production" Kibana is the one out on the CCS cluster, not on RED1. Xpack.ml.enabled is set to 'true' on every node in each cluster.

Any tips would be appreciated! Thank you!

Hey there @Datt_Mamon! :wave:

Does your ML node happen to have remote_cluster_client configured as well? Related docs.

Cheers!
Garrett

Just double-checked: yes, it does.

Okie, so I double checked with someone on the ML side, and apparently there is no notion of Remote ML Job's, just Remote Search Indices, which while not super clear, I suppose can be sussed out from this documentation:

If you want to run machine learning jobs, there must be at least one machine learning node in your cluster.

i.e. meaning you can only access ML Job's from Kibana nodes within the same cluster.

That said, it sounds like your CCS Kibana should be able to access the resulting indices from any ML Jobs/Transforms on the RED1 cluster, but you just can't do management actions from those other clusters. So if they need to be managed from your main CCS Kibana, you'll want to move the ML Node to that cluster.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.