First of all, I want to be clear that I looked at several guides and suggested similar questions before opening this post, but none of them worked for our case, so I am a bit desperate right now.
Here is our situation:
The developer who created all this configuration is gone and we don't have a way to reach him. We don't have any password for the certificates or anything, in case that is needed.
Last week, our Elasticsearch stopped adding new records as it reached the state where the disk was almost full. We changed the config to get some time and now is working as expected. However, we want to add a new server with elasticsearch and form a cluster as we don't want to resize the disk because we don't want to lose anything.
Here is the configuration for the main server /etc/elasticsearch/elasticsearch.yml
cluster.name: goulue
node.name: master-node
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 0.0.0.0
http.max_content_length: 100mb
discovery.seed_hosts: ["127.0.0.1", "ip_node"]
xpack.license.self_generated.type: "basic"
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
#xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.certificate: "certs/master-node/master-node.crt"
xpack.security.transport.ssl.certificate_authorities: "certs/ca/ca.crt"
#xpack.security.transport.ssl.truststore.path: "certs/elastic-certificates.p12"
xpack.security.http.ssl.enabled: true
#xpack.security.http.ssl.keystore.path: "certs/elastic-certificates.p12"
#xpack.security.http.ssl.truststore.path: "certs/elastic-certificates.p12"
xpack.security.http.ssl.certificate: certs/master-node/fullchain.crt
xpack.security.http.ssl.key: certs/master-node/privkey.key
#xpack.security.http.ssl.certificate_authorities: certs/ca/ca.crt
#
As I said, we created another server with the same elasticsearch version (7.6.2)
cluster.name: goulue
node.name: ico-elastic-node-2
node.data: true
node.master: false
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 0.0.0.0
discovery.zen.ping.unicast.hosts: ["ip_node_1"]
cluster.initial_master_nodes: ["master-node"]
xpack.security.transport.ssl.enabled: true
#xpack.security.transport.ssl.verification_mode: certificate
#xpack.security.transport.ssl.client_authentication: required
xpack.security.transport.ssl.certificate: master-node.crt
xpack.security.transport.ssl.certificate_authorities: ca.crt
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.certificate: fullchain.crt
xpack.security.http.ssl.key: privkey.key
I took a look at the logs and here is what I found:
Node 2
[ico-elastic-node-2] master not discovered yet: have discovered [{node-2}{G5SfrEv0RxaxmYf8urIFtQ}{XpfNNouCQx2HmYfw2AvoQw}{ip_server_2}{ip_server_29300}{dil}{ml.machine_memory=12558602240, xpack.installed=true, ml.max_open_jobs=20}]; discovery will continue using [ip_server_1, 127.0.0.1:9300] from hosts providers and [] from last-known cluster state; node term 0, last-accepted version 0 in term 0`
And the master:
[master-node] exception caught on transport layer [Netty4TcpChannel{localAddress=/ip:9300, remoteAddress=/ip:36660}], closing connection
io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: No available authentication scheme
Any suggestion? I'm desperate here because our master is yellow and we need to fix this ASAP.
The most important thing for us is to get the master node to green again and with enough space to make it work properly. If there is anything else we can do to avoid data lost without creating the cluster, please let me know.
Thank you for your time.
)