# New cluster setup fails

**URL:** https://discuss.elastic.co/t/new-cluster-setup-fails/378712
**Category:** Elasticsearch
**Created:** [May 29, 2025, 6:40pm UTC](https://discuss.elastic.co/t/new-cluster-setup-fails/378712 "2025-05-29T18:40:21Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)
#### Post date: [May 29, 2025, 6:40pm UTC](https://discuss.elastic.co/t/new-cluster-setup-fails/378712/1 "2025-05-29T18:40:21Z")

</div>

I have setup cluster in past without must problem. this time it is just failing don't understand why

Node leaves and joins fine as I test it out. but can't run reset password command for elastic

this is brand new cluster and I have no idea what the password is

/usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic  
output of this command is

```auto
Failed to determine the health of the cluster. Cluster health is currently RED.
This means that some cluster data is unavailable and your cluster is not fully functional.
The cluster logs (https://www.elastic.co/guide/en/elasticsearch/reference/8.16/logging.html) might contain information/indications for the underlying cause
It is recommended that you resolve the issues with your cluster before continuing
It is very likely that the command will fail when run against an unhealthy cluster.

If you still want to attempt to execute this command against an unhealthy cluster, you can pass the `-f` parameter.

ERROR: Failed to determine the health of the cluster. Cluster health is currently RED., with exit code 69

```

Everytime when I try to rest password following message shows up in log

```auto
[2025-05-29T13:28:23,940][INFO][o.e.x.s.a.f.FileUserRolesStore] [houelkdev11] users roles file [/etc/elasticsearch/users_roles] changed. updating users roles...
[2025-05-29T13:28:28,948][INFO][o.e.x.s.a.f.FileUserPasswdStore] [houelkdev11] users file [/etc/elasticsearch/users] changed. updating users...
[2025-05-29T13:28:28,951][INFO][o.e.x.s.a.f.FileUserRolesStore] [houelkdev11] users roles file [/etc/elasticsearch/users_roles] changed. updating users roles...

```

This file is empty and has proper user and permission

when I try with -f following shows up on elastic log  
it can't start up .ds-logs-deprecation index.

````auto
[2025-05-29T13:34:07,639][INFO][o.e.c.m.MetadataCreateIndexService] [houelkdev12] [.security-7] creating index, cause [api], templates [], shards [1]/[1]
[2025-05-29T13:34:07,650][INFO][o.e.c.r.a.AllocationService] [houelkdev12] updating number_of_replicas to [0] for indices [.security-7]
[2025-05-29T13:35:12,225][ERROR][o.e.x.d.l.DeprecationIndexingComponent] [houelkdev12] Bulk write of deprecation logs encountered some failures: [[lklSHZcBTejLPQKNwZBn org.elasticsearch.action.UnavailableShardsException: [.ds-.logs-deprecation.elasticsearch-default-2025.05.29-000001][0] primary shard is not active Timeout: [1m], request: [BulkShardRequest [[.ds-.logs-deprecation.elasticsearch-default-2025.05.29-000001][0]] containing [index {[.logs-deprecation.elasticsearch-default][lklSHZcBTejLPQKNwZBn], source[{"@timestamp":"2025-05-29T18:34:07.165Z", "log.level": "WARN", "data_stream.dataset":"deprecation.elasticsearch","data_stream.namespace":"default","data_stream.type":"logs","elasticsearch.event.category":"indices","event.code":"dot-prefix","message":"Index [.security-7] name begins with a dot (.), which is deprecated, and will not be allowed in a future Elasticsearch version." , "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"deprecation.elasticsearch","process.thread.name":"elasticsearch[houelkdev12][transport_worker][T#5]","log.logger":"org.elasticsearch.deprecation.validation.DotPrefixValidator","elasticsearch.cluster.uuid":"MISZKmNHS8qOwW22csVrDA","elasticsearch.node.id":"crnKNEoVRieZtnHmCBrt7g","elasticsearch.node.name":"houelkdev12","elasticsearch.cluster.name":"HouElkProd_dev"}
]}]]]]```
````

---

<div class="post-metadata">

### Author: ![RainTown](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raintown/32/140206_2.png) [@RainTown](https://discuss.elastic.co/u/RainTown)
#### Post date: [May 29, 2025, 6:48pm UTC](https://discuss.elastic.co/t/new-cluster-setup-fails/378712/2 "2025-05-29T18:48:57Z")

</div>

> [@elasticforme](#):
>
> This file is empty and has proper user and permission

Whats output of `ls -l /etc/elasticsearch/users`?

But

> [@elasticforme](#):
>
> this is brand new cluster and I have no idea what the password is

If nothing to lose, just start again? How are you installing? When you install it usually has a screenful of information at rpm/deb install time where it tells you the generated elastic password.

---

<div class="post-metadata">

### Author: ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)
#### Post date: [May 29, 2025, 6:53pm UTC](https://discuss.elastic.co/t/new-cluster-setup-fails/378712/3 "2025-05-29T18:53:33Z")

</div>

/etc/elasticsearch/users is blank file

I did restarted many time over by deleting whole data path dir, deleting elasticsearch.keystore file. Deleted everything from /etc/elasticsearch as welll.

this is debian, just use dpkg -i

here is my config. I have been using this certificate to other cluster without issue.

```auto
cluster.name: HouElkProd_dev
node.name: houelkdev11
node.roles: [master,ingest,transform]

path.data: /s1/elasticsearch
path.logs: /s1/log/elasticsearch
network.host: 10.21.226.93
http.port: 9200
discovery.seed_hosts: ["houelkdev11","houelkdev12","houekdev13"]
discovery.find_peers_interval: 5s
path.repo: /snapshot
action.destructive_requires_name: true

xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /etc/elasticsearch/config/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /etc/elasticsearch/config/elastic-certificates.p12

```

---

<div class="post-metadata">

### Author: ![RainTown](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raintown/32/140206_2.png) [@RainTown](https://discuss.elastic.co/u/RainTown)
#### Post date: [May 29, 2025, 8:55pm UTC](https://discuss.elastic.co/t/new-cluster-setup-fails/378712/4 "2025-05-29T20:55:46Z")

</div>

> [@elasticforme](#):
>
> /etc/elasticsearch/users is blank file

The reason I asked is I opened as bug when some file in /etc/elasticsearch/... got wrong permission, lost group-writeable bit, which led to issues. I dont have Debian, but on my Ubuntu:

```auto
$ sudo ls -l /etc/elasticsearch | grep users
-rw-rw---- 1 root elasticsearch 0 Jan 2 19:55 users
-rw-rw---- 1 root elasticsearch 0 Jan 2 19:55 users_roles

```

which is correct.

> [@elasticforme](#):
>
> I did restarted many time over by deleting whole data path dir, deleting elasticsearch.keystore file. Deleted everything from /etc/elasticsearch as welll.
> 
> this is debian, just use dpkg -I

Again, I would anticipate that on a blank system the default elastic user password is set and displayed in the post install. In fact I just checked, the 8.18.1 postinst script has sec ion shown below - something here is not adding up.

> node.roles: [master,ingest,transform]

Someone should maybe have a data role ?

```auto
$ ls -l elasticsearch_8.18.1_amd64.deb
-rw-r--r-- 1 root root 647923260 May 6 12:22 elasticsearch_8.18.1_amd64.deb
$ sha256sum elasticsearch_8.18.1_amd64.deb
11f78bcf5fab47afedd233ea0abdff1584c0ee7238b012d9ba99bd98f5cae96d elasticsearch_8.18.1_amd64.deb
$ dpkg-deb -R ~/elasticsearch_8.18.1_amd64.deb tmp
$ less tmp/DEBIAN/postinst
...
if ["x$IS_UPGRADE" != "xtrue"]; then
    # Don't exit immediately on error, we want to hopefully print some helpful banners
    set +e
    # Attempt to auto-configure security, this seems to be an installation
    if CLI_NAME="auto-configure-node" \
    CLI_LIBS="modules/x-pack-core,modules/x-pack-security,lib/tools/security-cli" \
    /usr/share/elasticsearch/bin/elasticsearch-cli <<< ""; then
        chown root:elasticsearch "${ES_PATH_CONF}"/certs/http.p12
        chown root:elasticsearch "${ES_PATH_CONF}"/certs/http_ca.crt
        chown root:elasticsearch "${ES_PATH_CONF}"/certs/transport.p12
        if INITIAL_PASSWORD=$(CLI_NAME=auto-config-gen-passwd \
        CLI_LIBS="modules/x-pack-core,modules/x-pack-security,lib/tools/security-cli" \
        /usr/share/elasticsearch/bin/elasticsearch-cli); then
            echo "--------------------------- Security autoconfiguration information ------------------------------"
            echo
            echo "Authentication and authorization are enabled."
            echo "TLS for the transport and HTTP layers is enabled and configured."
            echo
            echo "The generated password for the elastic built-in superuser is : ${INITIAL_PASSWORD}"
            echo
            echo "If this node should join an existing cluster, you can reconfigure this with"
            echo "'/usr/share/elasticsearch/bin/elasticsearch-reconfigure-node --enrollment-token <token-here>'"
            echo "after creating an enrollment token on your existing cluster."
            echo
            echo "You can complete the following actions at any time:"
            echo
            echo "Reset the password of the elastic built-in superuser with "
            echo "'/usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic'."
            echo
            echo "Generate an enrollment token for Kibana instances with "
            echo " '/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana'."
            echo
            echo "Generate an enrollment token for Elasticsearch nodes with "
            echo "'/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s node'."
            echo
            echo "-------------------------------------------------------------------------------------------------"
        fi

```

---

<div class="post-metadata">

### Author: ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)
#### Post date: [May 30, 2025, 5:38pm UTC](https://discuss.elastic.co/t/new-cluster-setup-fails/378712/5 "2025-05-30T17:38:42Z")

</div>

> [@RainTown](#):
>
> Someone should maybe have a data role ?

yep LOL LOL. how did I missed it. it is working now.

Thank you
