Can't see aws.cloudtrail logs in "Discover", but still getting Security Detections that uses aws.cloudtrail.
Keep on restarting the agent that ships the AWS Cloudtrail logs, okta logs, Google Workspace logs, but AWS eventually stop showing up in Discover after a little bit. This is getting quite annoying. This has never happened to me before.
AWS Execution via System Manager
Discover - Missing "aws.cloudtrail" since early on Feb 24th
Keep getting the following error in the Agent that ships the logs
[
"elastic_agent.filebeat"
][
"warn"
]"Cannot index event publisher.Event"{
"Content":"beat.Event"{
"Timestamp":time.Date(2022,
"time.February",
25,
20,
38,
14,
559510835,
"time.UTC)",
"Meta":{
"_id":"988637cc98-000000971422",
"raw_index":"logs-aws.cloudtrail-default"
},
"Fields":{
"agent":{
"ephemeral_id":"6537be7e-6022-4446-a94d-675a4b49d7d0",
"hostname":"<REDACTED>",
"id":"4e73bb02-1279-48c3-80e1-d515cd949a4a",
"name":"<REDACTED>",
"type":"filebeat",
"version":"7.17.0"
},
"aws":{
"s3":{
"bucket":{
"arn":"<REDACTED>",
"name":"<REDACTED>"
},
"object":{
"key":"<REDACTED>"
}
}
},
"cloud":{
"provider":"aws",
"region":"us-east-1"
},
"data_stream":{
"dataset":"aws.cloudtrail",
"namespace":"default",
"type":"logs"
},
"ecs":{
"version":"1.12.0"
},
"elastic_agent":{
"id":"4e73bb02-1279-48c3-80e1-d515cd949a4a",
"snapshot":false,
"version":"7.17.0"
},
"event":{
"dataset":"aws.cloudtrail"
},
"input":{
"type":"aws-s3"
},
"log":{
"file":{
"path":"<REDACTED>"
},
"offset":971422
},
"message":"{\"eventVersion\":\"1.08\",\"userIdentity\":{\"type\":\"AWSService\",\"invokedBy\":\"ecs.application-autoscaling.amazonaws.com\"},\"eventTime\":\"2022-02-24T07:18:56Z\",\"eventSource\":\"sts.amazonaws.com\",\"eventName\":\"AssumeRole\",\"awsRegion\":\"ap-southeast-2\",\"sourceIPAddress\":\"ecs.application-autoscaling.amazonaws.com\",\"userAgent\":\"ecs.application-autoscaling.amazonaws.com\",\"requestParameters\":{\"roleArn\":\"<REDACTED>\",\"roleSessionName\":\"<REDACTED>\",\"durationSeconds\":900},\"responseElements\":{\"credentials\":{\"accessKeyId\":\"<REDACTED>\",\"expiration\":\"Feb 24, 2022 7:33:56 AM\",\"sessionToken\":\"<REDACTED>\"},\"assumedRoleUser\":{\"assumedRoleId\":\"<REDACTED>\",\"arn\":\"<REDACTED>\"}},\"requestID\":\"<REDACTED>\",\"eventID\":\"<REDACTED>\",\"readOnly\":true,\"resources\":[{\"accountId\":\"<REDACTED>\"}],\"eventType\":\"AwsApiCall\",\"managementEvent\":true,\"recipientAccountId\":\"623976975596\",\"sharedEventID\":\"<REDACTED>\",\"eventCategory\":\"Management\"}",
"tags":[
"forwarded",
"aws-cloudtrail"
]
},
"Private":(*awss3.eventACKTracker)(0xc001f94870),
"TimeSeries":false
},
"Flags":0x1,
"Cache":"publisher.EventCache"{
"m":"common.MapStr(nil)"
}
}"(status=400)":{
"type":"illegal_argument_exception",
"reason":"pipeline with id [logs-aws.cloudtrail-1.9.0] does not exist"
},
"dropping event!"
