Collect logs from Amazon CloudWatch with Elastic Agent

[spo]

Hi
I am trying to pull cloudwatch logs with elastic agent.
I have elastic agent running on a windows server. It has the default policy attached to it. To the default policy I have added AWS integration , iis-logs integration and systemlogs integration . To Collect AWS CloudWatch logs using cloudwatch input, I have specified the access key ID and secret access key, log group ARN , log group name , region name.
The agent host status is healthy . It is pulling the iis logs and system logs fine, but not the cloudwatch logs.
I suspect it is something to do with permissions to pull the logs , but unable to figure out the exact error.
The IAM user that I am using to pull the logs has the following permissions
AmazonRDSFullAccess
AmazonEC2FullAccess
AmazonSQSFullAccess
IAMFullAccess
AmazonS3FullAccess
CloudWatchFullAccess
ResourceGroupsandTagEditorFullAccess
AmazonSNSFullAccess

[spo]

filebeat-json

{"file.name":"awscloudwatch/input.go","file.line":304},"message":"forwardEvent failed: OnEvent returned false. Stopping input worker","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2022-04-18T21:31:45.271Z","log.logger":"aws-cloudwatch","log.origin":{"file.name":"awscloudwatch/input.go","file.line":245},"message":"processLogEvents failed: forwardEvent failed: OnEvent returned false. Stopping input worker","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2022-04-18T21:31:45.812Z","log.logger":"aws-cloudwatch","log.origin":{"file.name":"awscloudwatch/input.go","file.line":304},"message":"forwardEvent failed: OnEvent returned false. Stopping input worker","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2022-04-18T21:31:45.812Z","log.logger":"aws-cloudwatch","log.origin":{"file.name":"awscloudwatch/input.go","file.line":245},"message":"processLogEvents failed: forwardEvent failed: OnEvent returned false. Stopping input worker","service.name":"filebeat","ecs.version":"1.6.0"}

[spo]

filebeat-json.log

{"log.level":"warn","@timestamp":"2022-04-18T19:58:42.106Z","log.logger":"elasticsearch","log.origin":{"file.name":"elasticsearch/client.go","file.line":414},"message":"Cannot index event publisher.Event{Content:beat.Event{Timestamp:time.Date(2022, time.February, 3, 21, 52, 1, 0, time.UTC), Meta:{\"_id\":\"*****\",\"raw_index\":\"logs-generic-default\"}, Fields:{\"agent\":{\"ephemeral_id\":\"*****\",\"hostname\":\"XYZ\",\"id\":\"123\",\"name\":\"XYZ\",\"type\":\"filebeat\",\"version\":\"7.17.2\"},\"awscloudwatch\":{\"ingestion_time\":\"2022-02-03T21:54:37.000Z\",\"log_group\":\"ABC\",\"log_stream\":\"/aws/network-firewall/flow/ABC_2022-02-03-21\"},\"cloud\":{\"provider\":\"aws\",\"region\":\"***\"},\"data_stream\":{\"dataset\":\"generic\",\"namespace\":\"default\",\"type\":\"logs\"},\"ecs\":{\"version\":\"1.12.0\"},\"elastic_agent\":{\"id\":\"****\",\"snapshot\":false,\"version\":\"7.17.2\"},\"event\":{\"dataset\":\"generic\",\"id\":\"****\",\"ingested\":\"2022-04-18T19:58:38.088Z\"},\"input\":{\"type\":\"aws-cloudwatch\"},\"log.file.path\":\"ABC//aws/network-firewall/flow/*****_2022-02-03-21\",\"message\":\"{\\\"firewall_name\\\":\\\"*****\\\",\\\"availability_zone\\\":\\\"****\\\",\\\"event_timestamp\\\":\\\"1643925121\\\",\\\"event\\\":{\\\"timestamp\\\":\\\"2022-02-03T21:52:01.127037+0000\\\",\\\"flow_id\\\":****,\\\"event_type\\\":\\\"netflow\\\",\\\"src_ip\\\":\\\"*******\\\",\\\"src_port\\\":*****,\\\"dest_ip\\\":\\\"******\\\",\\\"dest_port\\\":***,\\\"proto\\\":\\\"TCP\\\",\\\"app_proto\\\":\\\"tls\\\",\\\"netflow\\\":{\\\"pkts\\\":11,\\\"bytes\\\":1353,\\\"start\\\":\\\"2022-02-03T21:48:58.137493+0000\\\",\\\"end\\\":\\\"2022-02-03T21:48:59.171568+0000\\\",\\\"age\\\":1,\\\"min_ttl\\\":226,\\\"max_ttl\\\":226},\\\"tcp\\\":{\\\"tcp_flags\\\":\\\"1b\\\",\\\"syn\\\":true,\\\"fin\\\":true,\\\"psh\\\":true,\\\"ack\\\":true}}}\",\"tags\":[\"forwarded\",\"aws-cloudwatch-logs\"]}, Private:interface {}(nil), TimeSeries:false}, Flags:0x1, Cache:publisher.EventCache{m:common.MapStr(nil)}} (status=403): {\"type\":\"security_exception\",\"reason\":\"action [indices:admin/auto_create] is unauthorized for API key id [*******] of user [elastic/fleet-server] on indices [logs-generic-default], this action is granted by the index privileges [auto_configure,create_index,manage,all]\"}, dropping event!","service.name":"filebeat","ecs.version":"1.6.0"}