Thanks David, To include the changes, I had to bake my own ES docker Image. And while i need to have discovery in place for other containers to communicate with each other and appoint Master Node, I created elasticsearch.yml file and Dockerfile and build/deployed to ECS Fargate, but it is failing with errors as shown below:
{"type": "server", "timestamp": "2021-02-17T15:28:00,564Z", "level": "ERROR", "component": "o.e.d.e.Ec2DiscoveryPlugin", "cluster.name": "elasticsearch", "node.name": "ip-10-0-1-195.ap-south-1.compute.internal", "message": "failed to get metadata for [placement/availability-zone]",
{"type": "server", "timestamp": "2021-02-17T15:28:11,585Z", "level": "ERROR", "component": "o.e.b.ElasticsearchUncaughtExceptionHandler", "cluster.name": "elasticsearch", "node.name": "ip-10-0-1-195.ap-south-1.compute.internal", "message": "uncaught exception in thread [main]",
"stacktrace ["org.elasticsearch.bootstrap.StartupException: BindTransportException[Failed to resolve publish address]; nested: IOException[IOException caught when fetching InetAddress from [http://169.254.169.254/latest/meta-data/local-ipv4]]; nested: SocketException[Invalid argument];",
elasticsearch.yml
cluster.name: "elasticsearch"
path.data: /var/data/elasticsearch
path.logs: /var/logs/elasticsearch
bootstrap.memory_lock: false
network.host: 0.0.0.0
network.publish_host: _ec2:privateIp_
transport.publish_host: _ec2:privateIp_
discovery.zen.hosts_provider: ec2
discovery.ec2.tag.ElasticSearch: prod
discovery.ec2.endpoint: ec2.${REGION}.amazonaws.com
discovery.type: single-node
s3.client.default.endpoint: s3.${REGION}.amazonaws.com
cloud.node.auto_attributes: true
cluster.routing.allocation.awareness.attributes: aws_availability_zone
xpack.security.enabled: false
Dockerfile
FROM docker.elastic.co/elasticsearch/elasticsearch:7.11.0
ENV REGION us-east-1
ADD elasticsearch.yml /usr/share/elasticsearch/config/
USER root
RUN chown elasticsearch:elasticsearch config/elasticsearch.yml
USER elasticsearch
WORKDIR /usr/share/elasticsearch
RUN bin/elasticsearch-plugin install -b discovery-ec2 && bin/elasticsearch-plugin install -b repository-s3 && sed -e '/^-Xm/s/^/#/g' -i /usr/share/elasticsearch/config/jvm.options