# Filebeat Cloudtrail metricset breaks and stops returning "s3 log info" when upgrading past 7.9.1

**URL:** https://discuss.elastic.co/t/filebeat-cloudtrail-metricset-breaks-and-stops-returning-s3-log-info-when-upgrading-past-7-9-1/261302
**Category:** Beats
**Tags:** filebeat
**Created:** [January 16, 2021, 12:15am UTC](https://discuss.elastic.co/t/filebeat-cloudtrail-metricset-breaks-and-stops-returning-s3-log-info-when-upgrading-past-7-9-1/261302 "2021-01-16T00:15:56Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![AddChickpeas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/addchickpeas/32/44414_2.png) [@AddChickpeas](https://discuss.elastic.co/u/AddChickpeas)
#### Post date: [January 16, 2021, 12:15am UTC](https://discuss.elastic.co/t/filebeat-cloudtrail-metricset-breaks-and-stops-returning-s3-log-info-when-upgrading-past-7-9-1/261302/1 "2021-01-16T00:15:56Z")

</div>

I was attempting to upgrade filebeat to the latest release from 7.9.0, but am unable to get any version past 7.9.1 to work.

Starting with 7.9.2, debug shows filebeat receiving the SQS message, but it doesn't return any "s3 log info". It then deletes the sqs message.

```
 2021-01-16T00:05:08.828Z DEBUG [s3] s3/input.go:241 Processing 3 messages                                                                                                                                                 
 2021-01-16T00:05:08.828Z DEBUG [s3] s3/input.go:261 handleSQSMessage succeed and returned 0 sets of S3 log info                                                                                                           
 2021-01-16T00:05:08.828Z DEBUG [s3] s3/input.go:261 handleSQSMessage succeed and returned 0 sets of S3 log info                                                                                                           
 2021-01-16T00:05:08.828Z DEBUG [s3] s3/input.go:261 handleSQSMessage succeed and returned 0 sets of S3 log info                                                                                                           
 2021-01-16T00:05:08.828Z DEBUG [s3] s3/input.go:270 handleS3Objects succeed                                                                                                                                               
 2021-01-16T00:05:08.828Z DEBUG [s3] s3/input.go:270 handleS3Objects succeed        
 2021-01-16T00:05:08.828Z DEBUG [s3] s3/input.go:291 Deleting message from SQS: 0xc0002d6e00                                                                                                                               
 2021-01-16T00:05:08.828Z DEBUG [s3] s3/input.go:291 Deleting message from SQS: 0xc0002d6e00                                                                                                                               
 2021-01-16T00:05:08.828Z DEBUG [s3] s3/input.go:291 Deleting message from SQS: 0xc0002d6fb0                                                                                                                               
 2021-01-16T00:05:08.828Z DEBUG [s3] s3/input.go:291 Deleting message from SQS: 0xc0002d6ef0                                                                                                                               

```

With 7.9.1, it properly breaks down the message and publishes the event

```
 2021-01-16T00:14:30.424Z DEBUG [processors] processing/processors.go:187 Publish event: {                                                                                                                                 
   "@timestamp": "2021-01-16T00:14:30.424Z",                                                                                                                                                                                           
   "@metadata": {                                                                                                                                                                                                                      
     "beat": "filebeat",                                                                                                                                                                                                               
     "type": "_doc",                                                                                                                                                                                                                   
     "version": "7.9.1",                                                                                                                                                                                                               
     "_id": "4d35b4e3d4-000000017417",                                                                                                                                                                                                 
     "pipeline": "filebeat-7.9.1-aws-cloudtrail-pipeline"                                                                                                                                                                              
   },
```

---

<div class="post-metadata">

### Author: ![mtojek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mtojek/32/63863_2.png) [@mtojek](https://discuss.elastic.co/u/mtojek)
#### Post date: [January 18, 2021, 9:01am UTC](https://discuss.elastic.co/t/filebeat-cloudtrail-metricset-breaks-and-stops-returning-s3-log-info-when-upgrading-past-7-9-1/261302/2 "2021-01-18T09:01:31Z")

</div>

Jus to be sure - could you check and post the body of such SQS message? I wonder if they are correct.

---

<div class="post-metadata">

### Author: ![AddChickpeas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/addchickpeas/32/44414_2.png) [@AddChickpeas](https://discuss.elastic.co/u/AddChickpeas)
#### Post date: [January 19, 2021, 12:23am UTC](https://discuss.elastic.co/t/filebeat-cloudtrail-metricset-breaks-and-stops-returning-s3-log-info-when-upgrading-past-7-9-1/261302/3 "2021-01-19T00:23:51Z")

</div>

The messages appear to be formatted correctly. Since it is working fine in 7.9.1, it seems unlikely to me that the logs themselves are too blame. The only change being made is to the version of Filebeat.

```auto
{
      "Records": [
        {
          "eventVersion": "2.1",
          "eventSource": "aws:s3",
          "awsRegion": "us-east-2",
          "eventTime": "2021-01-19T00:14:00.509Z",
          "eventName": "ObjectCreated:Put",
          "userIdentity": {
            "principalId": ":regionalDeliverySession"
          },
          "requestParameters": {
            "sourceIPAddress": ""
          },
          "responseElements": {
            "x-amz-request-id": "",
            "x-amz-id-2": "+/NbEuu/v6dlVP1Yd8"
          },
          "s3": {
            "s3SchemaVersion": "1.0",
            "configurationId": "",
            "bucket": {
              "name": "-cloudtrail-logs",
              "ownerIdentity": {
                "principalId": ""
              },
              "arn": "arn:aws:s3:::-cloudtrail--logs"
            },
            "object": {
              "key": "AWSLogs///CloudTrail/us-east-1/2021/01/19/_CloudTrail_us-east-.json.gz",
              "size": 5834,
              "eTag": "",
              "sequencer": ""
            }
          }
        }
      ]
    }

```

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 16, 2021, 2:24am UTC](https://discuss.elastic.co/t/filebeat-cloudtrail-metricset-breaks-and-stops-returning-s3-log-info-when-upgrading-past-7-9-1/261302/4 "2021-02-16T02:24:09Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
