ES Admin cluster and Logging Cluster?

Elastic Orchestration Elastic Cloud Enterprise (ECE)
1

I would like to ask question about the architecture described in document.
There are actually three types of allocators.

  1. ES Admin cluster (Green one)
  2. ES Logging cluster (Green one)
  3. General cluster (Dark Blue one)

What is the difference with above three allocators and how do you configure through the cloud ui?
Also how can I map search workload and logging workload described here to each allocators?

2

@YuWatanabe,

They are all similar allocators, insofar as there is only one allocator role that can be assigned in the Cloud UI (hosts on which ECE is installed). The difference is in their purpose, which is what the diagram represents.

For your own workloads, you add more allocators to handle the workload that you want your Elasticsearch clusters to handle (#3 in your list). You can add these allocators by scaling out your installation (see https://www.elastic.co/guide/en/cloud-enterprise/current/ece-add-capacity.html, the same link as the one I gave in your other post).

ECE also uses allocators to host the ES admin cluster and the logging cluster (#1 & #2 in your list). These are required by ECE, but they technically run on allocators just like your own Elasticsearch clusters. I believe you could modify these (e.g. move their nodes from one allocator to another) and I think the final step in https://www.elastic.co/guide/en/cloud-enterprise/current/ece-planning.html#ece-topology may actually move them. However, I would exercise some care in what you do to these clusters, as you could affect the admin console.

You can modify ECE for search and logging workloads in alpha4 as described here: https://www.elastic.co/guide/en/cloud-enterprise/current/ece-change-ratio.html. In alpha4, the change you make to the RAM-to-disk ratio affects the entire installation. Future releases might provide more flexibility.

I hope this helps,

Nik

3

@YuWatanabe to add to what Nik has described, the centralized logging cluster is not yet available in alpha4, but will be available in the beta release expected in a few weeks

4

@nrichers

Thank you for the detail reply.

I think things started to make sense for me. I appreciate if you could answer to my additional questions.

When I first installed the Bootstrap Node , there was a elasticsearch cluster called admin-console-elasticsearch .

  1. Does admin-console-elasticsearch both include admin and logging cluster? and only installed in first node?

  2. What does admin-cluster do?

  3. What does logging-cluster do?

1600×310 22.3 KB

Thank you for taking your time.

5

@uricohen

Looking forward for the beta release. I appreciate if the document includes information about centralized logging cluster.

6

@YuWatanabe,

You're welcome! To answer your additional questions:

Re: 1: The admin-console-elasticsearch cluster is just the "ES Admin Cluster" (#1 in your list at the top). It does not include the logging cluster (there is no logging cluster in alpha4, yet). In beta1, an additional cluster will visible in the Cloud UI, which will be the ES Logging Cluster (#2 in your list). Initially, when you install ECE on your first host, that is where these clusters will be added as well, but as they are clusters, their nodes can technically be moved.

Re: 2: ECE uses Elasticsearch to keep track of cluster metadata. We basically use our own product as part of the ECE admin console. (We have to store information about these clusters somewhere, so what better place than our own product?)

Re: 3: The logging cluster will be used to collect metrics on all clusters running in your ECE installation. There will be more information on this in the beta1 documentation, but it is not available yet. (I own the documentation for ECE.)

I hope this helps,

Nik

7

@nrichers

Thank you . Now its clear.