Elasticsearch Cluster setup with security 7.2

I have setup elasticsearch cluster, with three linux vm servers, its working fine without security, but if i am enabling xpack security i am getting following error,

And for your information i am using trail License in 7.2

TestCluster Log

[2019-07-29T15:05:11,747][INFO ][o.e.x.s.a.AuthenticationService] [192.168.108.14] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
[2019-07-29T15:05:11,748][INFO ][o.e.x.s.a.AuthenticationService] [192.168.108.14] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
[2019-07-29T15:05:11,904][INFO ][o.e.x.s.a.AuthenticationService] [192.168.108.14] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
[2019-07-29T15:05:14,038][INFO ][o.e.x.s.a.AuthenticationService] [192.168.108.14] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
[2019-07-29T15:05:14,357][INFO ][o.e.x.s.a.AuthenticationService] [192.168.108.14] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
[2019-07-29T15:05:15,010][INFO ][o.e.x.s.a.AuthenticationService] [192.168.108.14] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
[2019-07-29T15:05:16,014][INFO ][o.e.x.s.a.AuthenticationService] [192.168.108.14] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
[2019-07-29T15:05:16,965][INFO ][o.e.x.s.a.AuthenticationService] [192.168.108.14] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
[2019-07-29T15:05:17,742][WARN ][o.e.c.c.ClusterFormationFailureHelper] [192.168.108.14] master not discovered or elected yet, an election requires at least 2 nodes with ids from [BFpBlWBbRVuSsj0_NnFVIA, aRZEtrLJTVm0qP7Z88kcVQ, TRPaaDxLQFOXVWHVH1boRg], have discovered which is not a quorum; discovery will continue using [192.168.108.40:9300, 192.168.108.41:9300] from hosts providers and [{192.168.108.14}{BFpBlWBbRVuSsj0_NnFVIA}{xsnlitimRGSSXhmVSgkgGw}{192.168.108.14}{192.168.108.14:9300}{ml.machine_memory=8201109504, xpack.installed=true, ml.max_open_jobs=20}] from last-known cluster state; node term 15, last-accepted version 353 in term 15
[2019-07-29T15:05:17,973][INFO ][o.e.x.s.a.AuthenticationService] [192.168.108.14] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
[2019-07-29T15:05:18,092][INFO ][o.e.x.s.a.AuthenticationService] [192.168.108.14] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
[2019-07-29T15:05:18,092][INFO ][o.e.x.s.a.AuthenticationService] [192.168.108.14] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
[2019-07-29T15:05:32,619][INFO ][o.e.x.s.a.AuthenticationService] [192.168.108.14] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]

without security i checked cluster health status, it looks fine. please check below for curl output:

[root@server elasticsearch]# curl -X GET "192.168.108.14:9200/_cluster/health?pretty"
{
"cluster_name" : "TestCluster",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 3,
"number_of_data_nodes" : 3,
"active_primary_shards" : 12,
"active_shards" : 24,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 0,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 100.0
}
[root@server elasticsearch]#

Please don't post unformatted code, logs, or configuration as it's very hard to read.

Instead, paste the text and format it with </> icon or pairs of triple backticks (```), and check the preview window to make sure it's properly formatted before posting it. This makes it more likely that your question will receive a useful answer.

It would be great if you could update your post to solve this.

Please check Configuration

cluster.name: TestCluster
node.name: 192.168.108.14
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 192.168.108.14
discovery.seed_hosts:

  • 192.168.108.40
  • 192.168.108.41
  • 192.168.108.14
    cluster.initial_master_nodes:
  • 192.168.108.14
  • 192.168.108.40
  • 192.168.108.41
    xpack.security.enabled: true
    xpack.security.transport.ssl.enabled: true
    xpack.security.transport.ssl.verification_mode: full
    xpack.security.transport.ssl.keystore.path: config/certs/server.com.p12
    xpack.security.transport.ssl.truststore.path: config/certs/server.com.p12

Please check now

Thanks very much for your interest in Elasticsearch.

Please be patient in waiting for responses to your question and refrain from pinging multiple times asking for a response or opening multiple topics for the same question. This is a community forum, it may take time for someone to reply to your question. For more information please refer to the Community Code of Conduct specifically the section "Be patient".

If you are in need of a service with an SLA that covers response times for questions then you may want to consider talking to us about a subscription.

Please help me on this,

Thanks yon in advance

Someone will help you with your issue, soon, please refrain from posting additional messages. As I said above:

Please be patient in waiting for responses to your question and refrain from pinging multiple times asking for a response or opening multiple topics for the same question. This is a community forum, it may take time for someone to reply to your question. For more information please refer to the Community Code of Conduct specifically the section "Be patient".

I copy that section for your help:

This mostly applies to forums, mailing lists, and code contributions (i.e. asynchronous forms of communication). Communities are often built on volunteer time both from participants and organizers. It is possible that your question or code contribution or suggestion might not receive an immediate response. Be patient and consider the norms of the community. One reminder ping is welcome, many reminder pings in rapid succession are not a good display of patience. Similarly, posting the same question in multiple threads is frowned upon and should not be done.

Can you provide use with the relevant configuration from all your 3 nodes and logs from the 3 of them ? Aren't there any additional errors or warnings in any of the other nodes ?
I don't see enough information here to know what might have caused your issues.

The issue is that the cluster cannot form

[2019-07-29T15:05:17,742][WARN ][o.e.c.c.ClusterFormationFailureHelper] [192.168.108.14] master not discovered or elected yet, an election requires at least 2 nodes with ids from [BFpBlWBbRVuSsj0_NnFVIA, aRZEtrLJTVm0qP7Z88kcVQ, TRPaaDxLQFOXVWHVH1boRg], have discovered which is not a quorum; discovery will continue using [192.168.108.40:9300, 192.168.108.41:9300] from hosts providers and [{192.168.108.14}{BFpBlWBbRVuSsj0_NnFVIA}{xsnlitimRGSSXhmVSgkgGw}{192.168.108.14}{192.168.108.14:9300}{ml.machine_memory=8201109504, xpack.installed=true, ml.max_open_jobs=20}] from last-known cluster state; node term 15, last-accepted version 353 in term 15

but we need to see larger parts of your logs to understand why

I fixed it, i found issue anyway thanks

It's nice that you resolved your issue, but it would be great if you could take the time to write down a few words about what the issue was and how you resolved it. This will be really helpful for others that might have similar issues with you and will end up looking at this post in the future !

while we use ssl communication between elasticsearch nodes need to send http request also in secure mode

so need to enable ssl by add this three lines as well in the configuration

xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.keystore.path: config/certs/server.com.p12
xpack.security.https.ssl.truststore.path: config/certs/server.com.p12

And need to add password for both transport and http by below command

cd /usr/share/elasticsearch/
./bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password
./bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password
./bin/elasticsearch-keystore add xpack.security.http.ssl.keystore.secure_password
/bin/elasticsearch-keystore add xpack.security.http.ssl.truststore.secure_password

1 Like