# Elasticsearch on AWS ECS Fargate

**URL:** <https://discuss.elastic.co/t/elasticsearch-on-aws-ecs-fargate/264596>\
**Category:** Elasticsearch\
**Created:** [February 17, 2021, 3:44pm UTC](https://discuss.elastic.co/t/elasticsearch-on-aws-ecs-fargate/264596 "2021-02-17T15:44:27Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Agron\_Demiraj](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/agron_demiraj/32/84055_2.png) [@Agron\_Demiraj](https://discuss.elastic.co/u/Agron_Demiraj)\
**Post date:** [February 17, 2021, 3:44pm UTC](https://discuss.elastic.co/t/elasticsearch-on-aws-ecs-fargate/264596/1 "2021-02-17T15:44:27Z")

</div>

Hi. I apologize if this is a duplicate of another thread, but so far I did not see a similar case.

I am trying to deploy ES on ECS with default image from [docker.elastic.co/elasticsearch/elasticsearch:7.11.1](http://docker.elastic.co/elasticsearch/elasticsearch:7.11.1) but it is failing with error

```auto
ERROR: [3] bootstrap checks failed
[1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65535]
[2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[3]: the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured
ERROR: Elasticsearch did not exit normally - check the logs at /usr/share/elasticsearch/logs/docker-cluster.log 

```

My target is to run multiple nodes connected to EFS and load balanced by a Load Balancer, but if single node works that will also be fine.

Your help would be highly appreciated.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [February 17, 2021, 9:22pm UTC](https://discuss.elastic.co/t/elasticsearch-on-aws-ecs-fargate/264596/2 "2021-02-17T21:22:10Z")

</div>

Welcome!

Please read [Bootstrap Checks | Elasticsearch Reference [7.11] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/bootstrap-checks.html)

---

<div class="post-metadata">

**Author:** ![Agron\_Demiraj](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/agron_demiraj/32/84055_2.png) [@Agron\_Demiraj](https://discuss.elastic.co/u/Agron_Demiraj)\
**Post date:** [February 18, 2021, 9:57am UTC](https://discuss.elastic.co/t/elasticsearch-on-aws-ecs-fargate/264596/3 "2021-02-18T09:57:43Z")

</div>

Thanks David, To include the changes, I had to bake my own ES docker Image. And while i need to have discovery in place for other containers to communicate with each other and appoint Master Node, I created **elasticsearch.yml** file and **Dockerfile** and build/deployed to ECS Fargate, but it is failing with errors as shown below:

```auto
{"type": "server", "timestamp": "2021-02-17T15:28:00,564Z", "level": "ERROR", "component": "o.e.d.e.Ec2DiscoveryPlugin", "cluster.name": "elasticsearch", "node.name": "ip-10-0-1-195.ap-south-1.compute.internal", "message": "failed to get metadata for [placement/availability-zone]",

{"type": "server", "timestamp": "2021-02-17T15:28:11,585Z", "level": "ERROR", "component": "o.e.b.ElasticsearchUncaughtExceptionHandler", "cluster.name": "elasticsearch", "node.name": "ip-10-0-1-195.ap-south-1.compute.internal", "message": "uncaught exception in thread [main]",

"stacktrace ["org.elasticsearch.bootstrap.StartupException: BindTransportException[Failed to resolve publish address]; nested: IOException[IOException caught when fetching InetAddress from [http://169.254.169.254/latest/meta-data/local-ipv4]]; nested: SocketException[Invalid argument];",

```

**elasticsearch.yml**

```auto
cluster.name: "elasticsearch"
path.data: /var/data/elasticsearch
path.logs: /var/logs/elasticsearch
bootstrap.memory_lock: false
network.host: 0.0.0.0
network.publish_host: _ec2:privateIp_
transport.publish_host: _ec2:privateIp_
discovery.zen.hosts_provider: ec2
discovery.ec2.tag.ElasticSearch: prod
discovery.ec2.endpoint: ec2.${REGION}.amazonaws.com
discovery.type: single-node
s3.client.default.endpoint: s3.${REGION}.amazonaws.com
cloud.node.auto_attributes: true
cluster.routing.allocation.awareness.attributes: aws_availability_zone
xpack.security.enabled: false

```

**Dockerfile**

```auto
FROM docker.elastic.co/elasticsearch/elasticsearch:7.11.0
ENV REGION us-east-1
ADD elasticsearch.yml /usr/share/elasticsearch/config/
USER root
RUN chown elasticsearch:elasticsearch config/elasticsearch.yml
USER elasticsearch
WORKDIR /usr/share/elasticsearch
RUN bin/elasticsearch-plugin install -b discovery-ec2 && bin/elasticsearch-plugin install -b repository-s3 && sed -e '/^-Xm/s/^/#/g' -i /usr/share/elasticsearch/config/jvm.options

```

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [February 18, 2021, 10:33am UTC](https://discuss.elastic.co/t/elasticsearch-on-aws-ecs-fargate/264596/4 "2021-02-18T10:33:42Z")

</div>

```nohighlight
IOException caught when fetching InetAddress from [http://169.254.169.254/latest...

```

You're using `_ec2:privateIp_` as your publish host, which means "look up my address from the EC2 metadata service", but it appears that the EC2 metadata service is not accessible within this container. I think that makes sense, you're running this in ECS Fargate not directly in EC2, so the `discovery-ec2` plugin isn't going to work for you.

---

<div class="post-metadata">

**Author:** ![Agron\_Demiraj](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/agron_demiraj/32/84055_2.png) [@Agron\_Demiraj](https://discuss.elastic.co/u/Agron_Demiraj)\
**Post date:** [February 18, 2021, 11:53am UTC](https://discuss.elastic.co/t/elasticsearch-on-aws-ecs-fargate/264596/5 "2021-02-18T11:53:50Z")

</div>

In essence this **discovery-ec2** plugin should work for ecs fargate as well, but if it does not, Does Elasticsearch offer a workaround for this ?

I am being forced to test Single Node container without discovery and see the outcome.

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [February 18, 2021, 12:14pm UTC](https://discuss.elastic.co/t/elasticsearch-on-aws-ecs-fargate/264596/6 "2021-02-18T12:14:02Z")

</div>

> [@Agron\_Demiraj](#):
>
> In essence this **discovery-ec2** plugin should work for ecs fargate as well

I don't think it should, no, ECS Fargate has its own [DNS-based service discovery mechanism](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service-discovery.html) that you should use instead.

---

<div class="post-metadata">

**Author:** ![Agron\_Demiraj](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/agron_demiraj/32/84055_2.png) [@Agron\_Demiraj](https://discuss.elastic.co/u/Agron_Demiraj)\
**Post date:** [February 18, 2021, 3:36pm UTC](https://discuss.elastic.co/t/elasticsearch-on-aws-ecs-fargate/264596/7 "2021-02-18T15:36:45Z")

</div>

You are right David. ECS Fargate uses Service Discovery and i will need to consider to use it. For now I will launch Single Node.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 18, 2021, 3:37pm UTC](https://discuss.elastic.co/t/elasticsearch-on-aws-ecs-fargate/264596/8 "2021-03-18T15:37:06Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.