Hi Friends,
I am facing issue with ELK stack deployed on AWS ECS.
Whenever I am trying to connect enable x-pack feature, Elasticsearch and Kibana containers are not coming up. EC2 instance is behind Application load balancer.
Error in Target group of Elasticsearch is “service es-services (instance i-096a3b81dcdfbf1b9) (port 9200) is unhealthy in target-group TG-ES due to (reason Health checks failed with these codes: [401])” and error in Target group of Kibana “service kb-services (instance i-096a3b81dcdfbf1b9) (port 50186) is unhealthy in target-group kibana-tg due to (reason Health checks failed with these codes: [302])”.
Container log of Elasticsearch is “[2022-05-01T06:19:33,469][INFO ][o.e.x.m.p.NativeController] [qlEfQvw] Native controller process has stopped - no new native processes can be started
2022-05-01 08:19:33
[2022-05-01T06:19:33,469][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [qlEfQvw] [controller/98] [Main.cc@148] Ml controller exiting
[2022-05-01T06:19:33,469][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [qlEfQvw] [controller/98] [Main.cc@148] Ml controller exiting
2022-05-01 08:19:33
[2022-05-01T06:19:33,374][INFO ][o.e.x.w.WatcherService ] [qlEfQvw] stopping watch service, reason [shutdown initiated]
[2022-05-01T06:19:33,374][INFO ][o.e.x.w.WatcherService ] [qlEfQvw] stopping watch service, reason [shutdown initiated]
”
Elasticsearch config is below
cluster.name: "prod"
bootstrap.memory_lock: false
network.host: 0.0.0.0
network.publish_host: ec2:privateIp
transport.publish_host: ec2:privateIp
discovery.zen.hosts_provider: ec2
discovery.ec2.tag.Elasticsearch: prod
discovery.ec2.endpoint: ec2.${REGION}.amazonaws.com
s3.client.default.endpoint: s3.${REGION}.amazonaws.com
cloud.node.auto_attributes: true
cluster.routing.allocation.awareness.attributes: aws_availability_zone
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.client_authentication: optional
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12
Elasticsearch version is 6.8.9
Thanks in advance for looking into this.
Best,
Anubhav
