# AWS logging integration error

**URL:** <https://discuss.elastic.co/t/aws-logging-integration-error/343469>\
**Category:** Elastic Agent\
**Created:** [September 20, 2023, 2:35pm UTC](https://discuss.elastic.co/t/aws-logging-integration-error/343469 "2023-09-20T14:35:11Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![rachelyang](https://avatars.discourse-cdn.com/v4/letter/r/51bf81/32.png) [@rachelyang](https://discuss.elastic.co/u/rachelyang)\
**Post date:** [September 20, 2023, 2:35pm UTC](https://discuss.elastic.co/t/aws-logging-integration-error/343469/1 "2023-09-20T14:35:11Z")

</div>

Hi,  
I want to integrate the AWS logs into my Kibana Observability log stream.  
Here is the access policy of SQS queue:

```auto
{
  "Version": "2012-10-17",
  "Id": "arn:aws:sqs:us-east-1:334811116626:ingest-ec2-logs-sqs/SQSDefaultPolicy",
  "Statement": [
    {
      "Sid": "Sid334811116626",
      "Effect": "Allow",
      "Principal": {
        "Service": "s3.amazonaws.com"
      },
      "Action": [
        "sqs:SendMessage",
        "sqs:ReceiveMessage"
      ],
      "Resource": "arn:aws:sqs:us-east-1:334811116626:ingest-ec2-logs-sqs"
    }
  ]
}

```

I set the SQS queue as the S3 event notification destination. Here is the S3 bucket event notification setting:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/1/3/138c7c1d4a74326ad23bfd8d763d60464fa1caff.png)  
I also configured integration in Kibana. I entered the SQS queue URL under the "Collect EC2 logs from S3" category.  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/5/9/594b7c3dbad0128ad3525624f386e6d8a617e410.png)  
I installed elastic agent on my elasticsearch server. The data stream in Kibana is empty. Here is the error in the elastic agent log:

```auto
{"log.level":"error","@timestamp":"2023-09-20T13:57:02.888Z","message":"failed to perform any bulk index operations: 500 Internal Server Error: {\"error\":{\"root_cause\":[{\"type\":\"illegal_state_exception\",\"reason\":\"There are no ingest nodes in this cluster, unable to forward request to an ingest node.\"}],\"type\":\"illegal_state_exception\",\"reason\":\"There are no ingest nodes in this cluster, unable to forward request to an ingest node.\"},\"status\":500}","component":{"binary":"metricbeat","dataset":"elastic_agent.metricbeat","id":"http/metrics-monitoring","type":"http/metrics"},"log":{"source":"http/metrics-monitoring"},"log.logger":"elasticsearch","log.origin":{"file.line":258,"file.name":"elasticsearch/client.go"},"service.name":"metricbeat","ecs.version":"1.6.0","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2023-09-20T13:57:02.979Z","message":"failed to publish events: 500 Internal Server Error: {\"error\":{\"root_cause\":[{\"type\":\"illegal_state_exception\",\"reason\":\"There are no ingest nodes in this cluster, unable to forward request to an ingest node.\"}],\"type\":\"illegal_state_exception\",\"reason\":\"There are no ingest nodes in this cluster, unable to forward request to an ingest node.\"},\"status\":500}","component":{"binary":"filebeat","dataset":"elastic_agent.filebeat","id":"filestream-monitoring","type":"filestream"},"log":{"source":"filestream-monitoring"},"log.logger":"publisher_pipeline_output","log.origin":{"file.line":174,"file.name":"pipeline/client_worker.go"},"service.name":"filebeat","ecs.version":"1.6.0","ecs.version":"1.6.0"}

```

How can I fix this issue?  
Thanks

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [September 20, 2023, 3:49pm UTC](https://discuss.elastic.co/t/aws-logging-integration-error/343469/2 "2023-09-20T15:49:29Z")

</div>

> [@rachelyang](#):
>
> `There are no ingest nodes in this cluster, unable to forward request to an ingest node.`

This is your error, it says that your cluster does not have any node with the `ingest` role.

Can you share the response of the following request in Kibana Dev Tools?

```auto
GET _cat/nodes?s=name

```

---

<div class="post-metadata">

**Author:** ![rachelyang](https://avatars.discourse-cdn.com/v4/letter/r/51bf81/32.png) [@rachelyang](https://discuss.elastic.co/u/rachelyang)\
**Post date:** [September 20, 2023, 3:55pm UTC](https://discuss.elastic.co/t/aws-logging-integration-error/343469/3 "2023-09-20T15:55:17Z")

</div>

I have two nodes in the elasticsearch cluster.  
10.0.9.12 46 96 2 1.51 2.36 1.93 dm \* cvelk5  
10.0.9.13 6 97 2 0.81 2.25 1.98 dm - cvelk6  
And this is the output configuration in elastic-agent.yml:

```auto
outputs:
  default:
    type: elasticsearch
    hosts:
      - 'https://10.0.9.12:9200'
    ssl.ca_trusted_fingerprint: f5bed68b70e82d4df44168279aa67bcf127577a76e3e7b57f53855827e2d2964
    username: 'elastic'
    password: '***'

```

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [September 20, 2023, 4:04pm UTC](https://discuss.elastic.co/t/aws-logging-integration-error/343469/4 "2023-09-20T16:04:35Z")

</div>

> [@rachelyang](#):
>
> 10.0.9.12 46 96 2 1.51 2.36 1.93 dm \* cvelk5  
> 10.0.9.13 6 97 2 0.81 2.25 1.98 dm - cvelk6

Yeah, none of the nodes has the `ingest` role, you will need to add the `ingest` role as a role for at least one of the nodes in `elasticsearch.yml` and restart the node.

A node with the ingest role is required for the integrations to work.

You can check here for the [documentation about node roles](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-node.html).

---

<div class="post-metadata">

**Author:** ![rachelyang](https://avatars.discourse-cdn.com/v4/letter/r/51bf81/32.png) [@rachelyang](https://discuss.elastic.co/u/rachelyang)\
**Post date:** [September 20, 2023, 4:27pm UTC](https://discuss.elastic.co/t/aws-logging-integration-error/343469/5 "2023-09-20T16:27:30Z")

</div>

Thank you. I set the nodes as master nodes, because I thougt they already have got the full functions.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 18, 2023, 4:27pm UTC](https://discuss.elastic.co/t/aws-logging-integration-error/343469/6 "2023-10-18T16:27:48Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
