# Log ingestion to Elastic Cloud not working with ECS Fargate

**URL:** <https://discuss.elastic.co/t/log-ingestion-to-elastic-cloud-not-working-with-ecs-fargate/300762>\
**Category:** Logs\
**Created:** [March 26, 2022, 3:02pm UTC](https://discuss.elastic.co/t/log-ingestion-to-elastic-cloud-not-working-with-ecs-fargate/300762 "2022-03-26T15:02:55Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Gabriel\_Carvalho](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gabriel_carvalho/32/103556_2.png) [@Gabriel\_Carvalho](https://discuss.elastic.co/u/Gabriel_Carvalho)\
**Post date:** [March 26, 2022, 3:02pm UTC](https://discuss.elastic.co/t/log-ingestion-to-elastic-cloud-not-working-with-ecs-fargate/300762/1 "2022-03-26T15:02:55Z")

</div>

Hi everyone!

I am trying to send logs of my apps running on an ECS Fargate Cluster to Elastic Cloud. I am using aws firelens logging driver and fluentbit as log router, I followed Elastic Cloud's documentation and everything seemed to be pretty straightforward, but it just doesn’t work...

In the first attempt, I was getting these errors:

```auto
2022-03-25 18:52:17[2022/03/25 21:52:17] [warn] [engine] chunk '1-1648245127.815032857.flb' cannot be retried: task_id=0, input=forward.1 > output=es.1
2022-03-25 18:52:11[2022/03/25 21:52:11] [error] [output:es:es.1] HTTP status=400 URI=/_bulk, response:
2022-03-25 18:52:11{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Action/metadata line [1] contains an unknown parameter [_type]"}],"type":"illegal_argument_exception","reason":"Action/metadata line [1] contains an unknown parameter [_type]"},"status":400}
2022-03-25 18:52:11[2022/03/25 21:52:11] [warn] [engine] failed to flush chunk '1-1648245127.815032857.flb', retry in 6 seconds: task_id=0, input=forward.1 > output=es.1 (out_id=1)
2022-03-25 18:51:36[2022/03/25 21:51:36] [info] [sp] stream processor started
2022-03-25 18:51:36[2022/03/25 21:51:36] [info] [input:forward:forward.2] listening on 127.0.0.1:24224
2022-03-25 18:51:36[2022/03/25 21:51:36] [info] [engine] started (pid=1)
2022-03-25 18:51:36[2022/03/25 21:51:36] [info] [storage] version=1.1.5, initializing...
2022-03-25 18:51:36[2022/03/25 21:51:36] [info] [storage] in-memory
2022-03-25 18:51:36[2022/03/25 21:51:36] [info] [storage] normal synchronization mode, checksum disabled, max_chunks_up=128
2022-03-25 18:51:36[2022/03/25 21:51:36] [info] [cmetrics] version=0.2.2
2022-03-25 18:51:36[2022/03/25 21:51:36] [info] [input:tcp:tcp.0] listening on 127.0.0.1:8877
2022-03-25 18:51:36[2022/03/25 21:51:36] [info] [input:forward:forward.1] listening on unix:///var/run/fluent.sock
2022-03-25 18:51:36e[1mFluent Bit v1.8.9e[0m
2022-03-25 18:51:36* e[1me[93mCopyright (C) 2019-2021 The Fluent Bit Authorse[0m
2022-03-25 18:51:36* e[1me[93mCopyright (C) 2015-2018 Treasure Datae[0m
2022-03-25 18:51:36* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
2022-03-25 18:51:36* https://fluentbit.io

```

After doing some search, I decided to enable Suppress\_Type\_Name option, like this: " **Suppress\_Type\_Name On**". After that, the errors above do not appear anymore, but fluentbit just doesn’t route any logs, like if it wasn’t able to read logs from the app:

```auto
2022-03-26 11:58:56[2022/03/26 14:58:56] [info] [sp] stream processor started
2022-03-26 11:58:56[2022/03/26 14:58:56] [info] [engine] started (pid=1)
2022-03-26 11:58:56[2022/03/26 14:58:56] [info] [storage] version=1.1.5, initializing...
2022-03-26 11:58:56[2022/03/26 14:58:56] [info] [storage] in-memory
2022-03-26 11:58:56[2022/03/26 14:58:56] [info] [storage] normal synchronization mode, checksum disabled, max_chunks_up=128
2022-03-26 11:58:56[2022/03/26 14:58:56] [info] [cmetrics] version=0.2.2
2022-03-26 11:58:56[2022/03/26 14:58:56] [info] [input:tcp:tcp.0] listening on 127.0.0.1:8877
2022-03-26 11:58:56[2022/03/26 14:58:56] [info] [input:forward:forward.1] listening on unix:///var/run/fluent.sock
2022-03-26 11:58:56[2022/03/26 14:58:56] [info] [input:forward:forward.2] listening on 127.0.0.1:24224
2022-03-26 11:58:56e[1mFluent Bit v1.8.9e[0m
2022-03-26 11:58:56* e[1me[93mCopyright (C) 2019-2021 The Fluent Bit Authorse[0m
2022-03-26 11:58:56* e[1me[93mCopyright (C) 2015-2018 Treasure Datae[0m
2022-03-26 11:58:56* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
2022-03-26 11:58:56* https://fluentbit.io

```

It keeps like this forever... This is my output config:

```auto
[OUTPUT]
    Name es
    Match app-name-firelens*
    Cloud_Auth ****
    Cloud_ID ****
    Include_Tag_Key true
    Index elastic_firelens
    Port 9243
    Suppress_Type_Name On
    Tag_Key tags
    tls On
    tls.verify Off

```

---

<div class="post-metadata">

**Author:** ![matschaffer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/matschaffer/32/95396_2.png) [@matschaffer](https://discuss.elastic.co/u/matschaffer)\
**Post date:** [April 4, 2022, 6:09am UTC](https://discuss.elastic.co/t/log-ingestion-to-elastic-cloud-not-working-with-ecs-fargate/300762/2 "2022-04-04T06:09:45Z")

</div>

hi @Gabriel_Carvalho!

Have you tried [AWS Fargate module | Filebeat Reference [8.1] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-awsfargate.html) ?

That might just work without having to traverse Fluent Bit.

Or if Fluent Bit is a requirement for you, I'd recommend also reaching out on their forums: [Discussions · fluent/fluentd · GitHub](https://github.com/fluent/fluentd/discussions)

[https://docs.fluentbit.io/manual/pipeline/outputs/elasticsearch](https://docs.fluentbit.io/manual/pipeline/outputs/elasticsearch) seems to say there should be some more logging information available if Elasticsearch isn't accepting the logs and the fluent bit community would be the best resource to find out how to get those logs.

---

<div class="post-metadata">

**Author:** ![Gabriel\_Carvalho](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gabriel_carvalho/32/103556_2.png) [@Gabriel\_Carvalho](https://discuss.elastic.co/u/Gabriel_Carvalho)\
**Post date:** [April 4, 2022, 6:23pm UTC](https://discuss.elastic.co/t/log-ingestion-to-elastic-cloud-not-working-with-ecs-fargate/300762/3 "2022-04-04T18:23:48Z")

</div>

Hi everyone, I found out that I was using the wrong data view. After creating a new data view filtering with `_index: elastic_firelens`, the logs came up.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 2, 2022, 6:24pm UTC](https://discuss.elastic.co/t/log-ingestion-to-elastic-cloud-not-working-with-ecs-fargate/300762/4 "2022-05-02T18:24:18Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.