Query on Cluster Creation, Endpoint and Proxies

[Ong (OCY)]

I have just finished setting up ECE and have a few questions on creating clusters and how to send search queries or send index writes to the clusters.

On the Create Cluster page, there is a drop down list of RAM and Storage for the nodes. Is it a must to have the amount of disk space available for the selected RAM size?

For each cluster, there is an Elasticsearch endpoint. Is the purpose to allow a remote Kibana to connect to that particular cluster?

I read from the ECE documentation that applications should connect to a cluster via the proxy. Are there examples on how to do so?

[zanbel (Roy Zanbel)]

Hi @Ong,

Good questions

On the Create Cluster page, there is a drop down list of RAM and Storage for the nodes. Is it a must to have the amount of disk space available for the selected RAM size?

You can change the RAM to storage ratio in order to better fit the underlying hardware you are using. You can follow these instructions in order to chage this setting.
We are also working on a new feature that will allow you to set the multiplier for each instance configuration, which is a new term that will be introduced, and support multiple multipliers for different node types, e.g. you can have different multipliers for different data nodes, or machine learning nodes, etc. More news will be shared shortly so stay tuned!

For each cluster, there is an Elasticsearch endpoint. Is the purpose to allow a remote Kibana to connect to that particular cluster?

This is the endpoint that you will use to ingest data to Elasticsearch through beats, Logstash or REST api, etc., or in case you need to query Elasticsearch directly for various reasons since not all use cases query Elasticsearch through Kibana.
Specifically around Kibana, yes, connecting another self managed Kibana instance is an example of why you would need to have that endpoint.

I read from the ECE documentation that applications should connect to a cluster via the proxy. Are there examples on how to do so?

The proxy layer specified in the architecture diagram in our user guide described a proxy layer that is managed by ECE itself, and is not something you will need to setup and manage outside of ECE.
This is one of the role you can enable, and we have a few suggestion for different playbooks that should help you understand how many runners should have the proxy role enabled based on your ECE deployment size and expected workload.
Related to this point, we do recommend to have load-balancer in front our ECE proxy layer which you will need to provide and maintain, but it is not mandatory.

Hope this helps.