Architecture Question

We have a 10 node cluster running 7.6 on s390x. I'm aware s390x is not supported but we've been able to make it work. We want to upgrade to 7.17.

With the 7.17 upgrade we want to have at least one node to use machine learning but we cannot get it to work on s390x. We also are struggling to get Kibana 7.17 to build on s390x.

So, our idea was to use a separate network with additional resources at our disposal that is not s390x based. Our plan was to set up a single node cluster on it and have that node be a machine learning node. We would have Kibana 7.17 connect to it and then use a remote cluster connection to connect to our s390x cluster.

After locally testing out remote clustering I'm now understanding what remote clustering is and isn't and I don't think this architecture will work.

Remote Clustering doesn't make two clusters appear as one as I had thought. For example if I do GET /_cat/indices , I only see the indices on the single node cluster and not the remote cluster.

So, with all that said is there a recommended architecture to keep an existing cluster but run Kibana and at least one node with machine learning on a different network due to differences in hardware?

After mulling on this, I wonder if it makes the most sense to have a Kibana instance on the new network connect to the s390x cluster as our current Kibana does and simply add a machine learning node to the cluster that is physically on a different network. That way we still have a single cluster.

Thank you for your help.

Hi @bsherman
Welcome to the community.

You are absolutely on the right track from what I can tell.

I think you missed somewhere that the indices can absolutely be accessed remotely through cross cluster search CCS and used in an ML job.

You just set up an index pattern with the remote syntax that looks like this.

*:my-index-*

That first * says get index from any remote host. You can name it as well.

remote-host:my-index-*

Then you can base a machine learning job on that index pattern or in discover or visualizations etc.

That's definitely a use case of CCS

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.