# Collect logs from Amazon CloudWatch with Elastic Agent **URL:** \ **Category:** Beats\ **Tags:** elastic-stack-monitoring, elastic-stack-security, fleet, elastic-agent\ **Created:** [April 18, 2022, 9:29pm UTC](https://discuss.elastic.co/t/collect-logs-from-amazon-cloudwatch-with-elastic-agent/302648 "2022-04-18T21:29:37Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![spo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spo/32/78055_2.png) [@spo](https://discuss.elastic.co/u/spo)\ **Post date:** [April 18, 2022, 9:29pm UTC](https://discuss.elastic.co/t/collect-logs-from-amazon-cloudwatch-with-elastic-agent/302648/1 "2022-04-18T21:29:37Z")
Hi I am trying to pull cloudwatch logs with elastic agent. I have elastic agent running on a windows server. It has the default policy attached to it. To the default policy I have added AWS integration , iis-logs integration and systemlogs integration . To Collect AWS CloudWatch logs using cloudwatch input, I have specified the access key ID and secret access key, log group ARN , log group name , region name. The agent host status is healthy . It is pulling the iis logs and system logs fine, but not the cloudwatch logs. I suspect it is something to do with permissions to pull the logs , but unable to figure out the exact error. The IAM user that I am using to pull the logs has the following permissions AmazonRDSFullAccess AmazonEC2FullAccess AmazonSQSFullAccess IAMFullAccess AmazonS3FullAccess CloudWatchFullAccess ResourceGroupsandTagEditorFullAccess AmazonSNSFullAccess ---
**Author:** ![spo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spo/32/78055_2.png) [@spo](https://discuss.elastic.co/u/spo)\ **Post date:** [April 18, 2022, 9:35pm UTC](https://discuss.elastic.co/t/collect-logs-from-amazon-cloudwatch-with-elastic-agent/302648/2 "2022-04-18T21:35:04Z")
filebeat-json ```auto {"file.name":"awscloudwatch/input.go","file.line":304},"message":"forwardEvent failed: OnEvent returned false. Stopping input worker","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"error","@timestamp":"2022-04-18T21:31:45.271Z","log.logger":"aws-cloudwatch","log.origin":{"file.name":"awscloudwatch/input.go","file.line":245},"message":"processLogEvents failed: forwardEvent failed: OnEvent returned false. Stopping input worker","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"error","@timestamp":"2022-04-18T21:31:45.812Z","log.logger":"aws-cloudwatch","log.origin":{"file.name":"awscloudwatch/input.go","file.line":304},"message":"forwardEvent failed: OnEvent returned false. Stopping input worker","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"error","@timestamp":"2022-04-18T21:31:45.812Z","log.logger":"aws-cloudwatch","log.origin":{"file.name":"awscloudwatch/input.go","file.line":245},"message":"processLogEvents failed: forwardEvent failed: OnEvent returned false. Stopping input worker","service.name":"filebeat","ecs.version":"1.6.0"} ``` ---
**Author:** ![spo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spo/32/78055_2.png) [@spo](https://discuss.elastic.co/u/spo)\ **Post date:** [April 18, 2022, 9:59pm UTC](https://discuss.elastic.co/t/collect-logs-from-amazon-cloudwatch-with-elastic-agent/302648/3 "2022-04-18T21:59:39Z")
filebeat-json.log ```auto {"log.level":"warn","@timestamp":"2022-04-18T19:58:42.106Z","log.logger":"elasticsearch","log.origin":{"file.name":"elasticsearch/client.go","file.line":414},"message":"Cannot index event publisher.Event{Content:beat.Event{Timestamp:time.Date(2022, time.February, 3, 21, 52, 1, 0, time.UTC), Meta:{\"_id\":\" *****\",\"raw_index\":\"logs-generic-default\"}, Fields:{\"agent\":{\"ephemeral_id\":\"***** \",\"hostname\":\"XYZ\",\"id\":\"123\",\"name\":\"XYZ\",\"type\":\"filebeat\",\"version\":\"7.17.2\"},\"awscloudwatch\":{\"ingestion_time\":\"2022-02-03T21:54:37.000Z\",\"log_group\":\"ABC\",\"log_stream\":\"/aws/network-firewall/flow/ABC_2022-02-03-21\"},\"cloud\":{\"provider\":\"aws\",\"region\":\" ***\"},\"data_stream\":{\"dataset\":\"generic\",\"namespace\":\"default\",\"type\":\"logs\"},\"ecs\":{\"version\":\"1.12.0\"},\"elastic_agent\":{\"id\":\"**** \",\"snapshot\":false,\"version\":\"7.17.2\"},\"event\":{\"dataset\":\"generic\",\"id\":\" ****\",\"ingested\":\"2022-04-18T19:58:38.088Z\"},\"input\":{\"type\":\"aws-cloudwatch\"},\"log.file.path\":\"ABC//aws/network-firewall/flow/***** _2022-02-03-21\",\"message\":\"{\\\"firewall_name\\\":\\\" *****\\\",\\\"availability_zone\\\":\\\"**** \\\",\\\"event_timestamp\\\":\\\"1643925121\\\",\\\"event\\\":{\\\"timestamp\\\":\\\"2022-02-03T21:52:01.127037+0000\\\",\\\"flow_id\\\": ****,\\\"event_type\\\":\\\"netflow\\\",\\\"src_ip\\\":\\\"******* \\\",\\\"src_port\\\": *****,\\\"dest_ip\\\":\\\"****** \\\",\\\"dest_port\\\":***,\\\"proto\\\":\\\"TCP\\\",\\\"app_proto\\\":\\\"tls\\\",\\\"netflow\\\":{\\\"pkts\\\":11,\\\"bytes\\\":1353,\\\"start\\\":\\\"2022-02-03T21:48:58.137493+0000\\\",\\\"end\\\":\\\"2022-02-03T21:48:59.171568+0000\\\",\\\"age\\\":1,\\\"min_ttl\\\":226,\\\"max_ttl\\\":226},\\\"tcp\\\":{\\\"tcp_flags\\\":\\\"1b\\\",\\\"syn\\\":true,\\\"fin\\\":true,\\\"psh\\\":true,\\\"ack\\\":true}}}\",\"tags\":[\"forwarded\",\"aws-cloudwatch-logs\"]}, Private:interface {}(nil), TimeSeries:false}, Flags:0x1, Cache:publisher.EventCache{m:common.MapStr(nil)}} (status=403): {\"type\":\"security_exception\",\"reason\":\"action [indices:admin/auto_create] is unauthorized for API key id [*******] of user [elastic/fleet-server] on indices [logs-generic-default], this action is granted by the index privileges [auto_configure,create_index,manage,all]\"}, dropping event!","service.name":"filebeat","ecs.version":"1.6.0"} ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 16, 2022, 10:00pm UTC](https://discuss.elastic.co/t/collect-logs-from-amazon-cloudwatch-with-elastic-agent/302648/4 "2022-05-16T22:00:18Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.