# Production deployment with dedicated masters

**URL:** <https://discuss.elastic.co/t/production-deployment-with-dedicated-masters/346965>\
**Category:** Elasticsearch\
**Created:** [November 13, 2023, 8:29am UTC](https://discuss.elastic.co/t/production-deployment-with-dedicated-masters/346965 "2023-11-13T08:29:33Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![DaddyYusk](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daddyyusk/32/127617_2.png) [@DaddyYusk](https://discuss.elastic.co/u/DaddyYusk)\
**Post date:** [November 13, 2023, 8:29am UTC](https://discuss.elastic.co/t/production-deployment-with-dedicated-masters/346965/1 "2023-11-13T08:29:33Z")

</div>

Hi,

I plan to deploy Elasticsearch for a production environment following this architecture :

- 3 x dedicated Master node (`node.roles: [master]`)
- 1 x Hot Data node (`node.roles: [data_hot]`)
- 1 x Warm Data node (`node.roles: [data_warm]`)
- 1 x Cold Data node (`node.roles: [data_cold]`)
- 1 x Fleet Server
- 1 x Kibana

My question is : do I also need "data", "data\_content", "ingest" and "transform" node roles and on which instance do I need to set these?

Thanks a lot for your help!

---

<div class="post-metadata">

**Author:** ![DaddyYusk](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daddyyusk/32/127617_2.png) [@DaddyYusk](https://discuss.elastic.co/u/DaddyYusk)\
**Post date:** [November 15, 2023, 4:21pm UTC](https://discuss.elastic.co/t/production-deployment-with-dedicated-masters/346965/2 "2023-11-15T16:21:37Z")

</div>

Hi,  
I encountered a problem straight from the beginning.

After the installation of my first dedicated Master node (`node.roles: [master]`) my cluster health seems red as I'm unable to reset password or generate token for adding new nodes.

So for the second node, I've setup the Hot Data node (`cluster.name: bla, node.roles: [data_hot, ingest, transform], cluster.initial_master_nodes: ["master-1"]`) without a token but this doesn't solve the problem...

So do I need to setup as the first node a classic elasticsearch without defining `node.roles` then when the whole architecture is deployed reconfigure it to a `master` only? But I will loose shard no?

Thanks for your help.  
Regards.

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [November 15, 2023, 6:11pm UTC](https://discuss.elastic.co/t/production-deployment-with-dedicated-masters/346965/3 "2023-11-15T18:11:59Z")

</div>

> [@DaddyYusk](#):
>
> My question is : do I also need "data", "data\_content", "ingest" and "transform" node roles and on which instance do I need to set these?

The `data` role is a generic role the can act as all the other data roles, since you are planning to have tiered data nodes, you do not need the generic one, but you need a `data_content` node, `data_hot` and `data_content` are required.

You may add the `data_content` role to the `data_hot` node.

Also, with just one node for each data tier you won't be able to have replicas.

Since you are planning to use fleet you also need to have an `ingest` node and a `transform` node.

You will probably need to change the roles of your `data_hot` node to this:

```auto
node.roles: [data_hot, data_content, ingest, transform]

```

> [@DaddyYusk](#):
>
> After the installation of my first dedicated Master node (`node.roles: [master]`) my cluster health seems red as I'm unable to reset password or generate token for adding new nodes.
> 
> So for the second node, I've setup the Hot Data node (`cluster.name: bla, node.roles: [data_hot, ingest, transform], cluster.initial_master_nodes: ["master-1"]`) without a token but this doesn't solve the problem...

You need the `data_content` role, as explained it is required, the system indices uses this role, this is also an [important note in the documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-node.html).

---

<div class="post-metadata">

**Author:** ![DaddyYusk](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daddyyusk/32/127617_2.png) [@DaddyYusk](https://discuss.elastic.co/u/DaddyYusk)\
**Post date:** [November 15, 2023, 7:20pm UTC](https://discuss.elastic.co/t/production-deployment-with-dedicated-masters/346965/4 "2023-11-15T19:20:07Z")

</div>

Hi @leandrojmp,

Many, many thanks for your advices and this explaination, that makes perfect sense!

I have one more question please :

- As my cluster health is red after the first master node installation (because lacking the `data_content` and `data_hot` roles), how to enroll in my cluster the Data\_Hot node without the ability to generate an enrollment token ?

Have a great day!  
Regards.

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [November 15, 2023, 8:23pm UTC](https://discuss.elastic.co/t/production-deployment-with-dedicated-masters/346965/5 "2023-11-15T20:23:56Z")

</div>

> [@DaddyYusk](#):
>
> As my cluster health is red after the first master node installation (because lacking the `data_content` and `data_hot` roles), how to enroll in my cluster the Data\_Hot node without the ability to generate an enrollment token ?

Did you configure all nodes and start them or you just configured one node and started it? You need to configure your 3 masters + your data hot node and start them.

---

<div class="post-metadata">

**Author:** ![DaddyYusk](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daddyyusk/32/127617_2.png) [@DaddyYusk](https://discuss.elastic.co/u/DaddyYusk)\
**Post date:** [November 16, 2023, 8:57am UTC](https://discuss.elastic.co/t/production-deployment-with-dedicated-masters/346965/6 "2023-11-16T08:57:23Z")

</div>

Hi @leandrojmp,

Once again many thanks for the advice!

What I was doing before :

- Installation and configuration of 1 master node
- Installation and configuration of 1 data hot node
- Start the master node, then start the data hot node just after.

What I am actually doing now :

- Installation and configuration of the 3 master nodes with the same yml file :

```auto
cluster.name: MyClusterProd
node.name: master-1 # Change for each master node
node.roles: [master]
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 192.168.74.30 # Change for each master node
http.port: 9200
xpack.security.enabled: true
xpack.security.enrollment.enabled: true
xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12
xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12
cluster.initial_master_nodes: ["master-1", "master-2", "master-3"]
http.host: 0.0.0.0
transport.host: 0.0.0.0

```

- Installation and configuration of the data hot node using this yml file :

```auto
cluster.name: MyClusterProd
node.name: datah-1
node.roles: [data_hot, data_content, ingest, transform]
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 192.168.74.42
http.port: 9200
xpack.security.enabled: true
xpack.security.enrollment.enabled: true
xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12
xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12
cluster.initial_master_nodes: ["master-1", "master-2", "master-3"]
http.host: 0.0.0.0

```

Despite following this documentation [Bootstrapping a cluster](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery-bootstrap-cluster.html), I don't figure out how and in which order to start each node of the cluster.  
Just remember that right after the startup of master-1, the cluster health is red and I'm unable to generate an enrollment token.

Have a great day.  
Regards.

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [November 16, 2023, 1:38pm UTC](https://discuss.elastic.co/t/production-deployment-with-dedicated-masters/346965/7 "2023-11-16T13:38:54Z")

</div>

> [@DaddyYusk](#):
>
> Just remember that right after the startup of master-1, the cluster health is red and I'm unable to generate an enrollment token.

I never used this enrollment token approach, but if I'm not wrong it works only on specfic scenarios, like your nodes will have all the roles and the certificate will be auto-generated.

The documentation for the bootstrapping a cluster assumes that your nodes will have all roles, so you will already have a bootstrapped cluster and could use the enrollment token, this is not your case since you want to have master dedicated and data dedicated nodes and you need nodes with `master`, `data_content` and `data_hot` roles.

My suggestion would be to configure your `data_hot`/`data_content` nodes to also be master nodes and start all the 5 nodes at the same time and wait for the cluster formation.

After that you could remove the master role for the data nodes and restart them.

---

<div class="post-metadata">

**Author:** ![DaddyYusk](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daddyyusk/32/127617_2.png) [@DaddyYusk](https://discuss.elastic.co/u/DaddyYusk)\
**Post date:** [November 16, 2023, 2:19pm UTC](https://discuss.elastic.co/t/production-deployment-with-dedicated-masters/346965/8 "2023-11-16T14:19:43Z")

</div>

Thanks a lot for the suggestion about giving to Data Hot node the master role for the initial cluster setup then remove it.

> [@leandrojmp](#):
>
> I never used this enrollment token approach

Could you please provide me what enrollment approach do you usually use?  
Thanks!

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [November 16, 2023, 2:48pm UTC](https://discuss.elastic.co/t/production-deployment-with-dedicated-masters/346965/9 "2023-11-16T14:48:27Z")

</div>

> [@DaddyYusk](#):
>
> Could you please provide me what enrollment approach do you usually use?

Basically you just need to create the certificates for all nodes and configure the `elasticsearch.yml` files, from what I understood the enrollment token only serves to simplify the certification generation, that's why it rely on you using the auto-generate certificates.

If you manually create certificates for your nodes, you won't need to use the enrollment token command, you just create the certificates, copy it to the nodes, configure the `elasticsearch.yml` and start the nodes.

---

<div class="post-metadata">

**Author:** ![DaddyYusk](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daddyyusk/32/127617_2.png) [@DaddyYusk](https://discuss.elastic.co/u/DaddyYusk)\
**Post date:** [November 17, 2023, 3:20pm UTC](https://discuss.elastic.co/t/production-deployment-with-dedicated-masters/346965/10 "2023-11-17T15:20:00Z")

</div>

Hi @leandrojmp,

> [@leandrojmp](#):
>
> My suggestion would be to configure your `data_hot`/`data_content` nodes to also be master nodes and start all the 5 nodes at the same time and wait for the cluster formation.
> 
> After that you could remove the master role for the data nodes and restart them.

Thanks a lot for this advice, it easily solved my problem!  
Now my cluster is up and running with 3 dedicated master nodes.

Have a great day.  
Regards.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 15, 2023, 3:20pm UTC](https://discuss.elastic.co/t/production-deployment-with-dedicated-masters/346965/11 "2023-12-15T15:20:46Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
