Can't see aws.cloudtrail logs in "Discover", but still getting Security Detections that uses aws.cloudtrail

Can't see aws.cloudtrail logs in "Discover", but still getting Security Detections that uses aws.cloudtrail.

Keep on restarting the agent that ships the AWS Cloudtrail logs, okta logs, Google Workspace logs, but AWS eventually stop showing up in Discover after a little bit. This is getting quite annoying. This has never happened to me before.

AWS Execution via System Manager

Discover - Missing "aws.cloudtrail" since early on Feb 24th

Keep getting the following error in the Agent that ships the logs

[
   "elastic_agent.filebeat"
][
   "warn"
]"Cannot index event publisher.Event"{
   "Content":"beat.Event"{
      "Timestamp":time.Date(2022,
      "time.February",
      25,
      20,
      38,
      14,
      559510835,
      "time.UTC)",
      "Meta":{
         "_id":"988637cc98-000000971422",
         "raw_index":"logs-aws.cloudtrail-default"
      },
      "Fields":{
         "agent":{
            "ephemeral_id":"6537be7e-6022-4446-a94d-675a4b49d7d0",
            "hostname":"<REDACTED>",
            "id":"4e73bb02-1279-48c3-80e1-d515cd949a4a",
            "name":"<REDACTED>",
            "type":"filebeat",
            "version":"7.17.0"
         },
         "aws":{
            "s3":{
               "bucket":{
                  "arn":"<REDACTED>",
                  "name":"<REDACTED>"
               },
               "object":{
                  "key":"<REDACTED>"
               }
            }
         },
         "cloud":{
            "provider":"aws",
            "region":"us-east-1"
         },
         "data_stream":{
            "dataset":"aws.cloudtrail",
            "namespace":"default",
            "type":"logs"
         },
         "ecs":{
            "version":"1.12.0"
         },
         "elastic_agent":{
            "id":"4e73bb02-1279-48c3-80e1-d515cd949a4a",
            "snapshot":false,
            "version":"7.17.0"
         },
         "event":{
            "dataset":"aws.cloudtrail"
         },
         "input":{
            "type":"aws-s3"
         },
         "log":{
            "file":{
               "path":"<REDACTED>"
            },
            "offset":971422
         },
         "message":"{\"eventVersion\":\"1.08\",\"userIdentity\":{\"type\":\"AWSService\",\"invokedBy\":\"ecs.application-autoscaling.amazonaws.com\"},\"eventTime\":\"2022-02-24T07:18:56Z\",\"eventSource\":\"sts.amazonaws.com\",\"eventName\":\"AssumeRole\",\"awsRegion\":\"ap-southeast-2\",\"sourceIPAddress\":\"ecs.application-autoscaling.amazonaws.com\",\"userAgent\":\"ecs.application-autoscaling.amazonaws.com\",\"requestParameters\":{\"roleArn\":\"<REDACTED>\",\"roleSessionName\":\"<REDACTED>\",\"durationSeconds\":900},\"responseElements\":{\"credentials\":{\"accessKeyId\":\"<REDACTED>\",\"expiration\":\"Feb 24, 2022 7:33:56 AM\",\"sessionToken\":\"<REDACTED>\"},\"assumedRoleUser\":{\"assumedRoleId\":\"<REDACTED>\",\"arn\":\"<REDACTED>\"}},\"requestID\":\"<REDACTED>\",\"eventID\":\"<REDACTED>\",\"readOnly\":true,\"resources\":[{\"accountId\":\"<REDACTED>\"}],\"eventType\":\"AwsApiCall\",\"managementEvent\":true,\"recipientAccountId\":\"623976975596\",\"sharedEventID\":\"<REDACTED>\",\"eventCategory\":\"Management\"}",
         "tags":[
            "forwarded",
            "aws-cloudtrail"
         ]
      },
      "Private":(*awss3.eventACKTracker)(0xc001f94870),
      "TimeSeries":false
   },
   "Flags":0x1,
   "Cache":"publisher.EventCache"{
      "m":"common.MapStr(nil)"
   }
}"(status=400)":{
   "type":"illegal_argument_exception",
   "reason":"pipeline with id [logs-aws.cloudtrail-1.9.0] does not exist"
},
"dropping event!"

Did u somehow delete the pipeline?

Nope. It's right here.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.