401 error when setting up filebeat google_workspace integration

Elastic Stack Beats
1

Hi All,

I'm using filebeat (7.17.9) and trying to setup google workspace integration.

I've followed all the steps in:

I've got a json credential file:

{
  "type": "service_account",
  "project_id": "gwm-1688565370135",
  "private_key_id": "xxx",
  "private_key": "xxx",
  "client_email": "gwm-service-account@gwm-xxx.iam.gserviceaccount.com",
  "client_id": "xxx",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/gwm-service-account%40gwm-1688565370135.iam.gserviceaccount.com",
  "universe_domain": "googleapis.com"
}

This is the config file:

# Module: google_workspace
# Docs: https://www.elastic.co/guide/en/beats/filebeat/7.17/filebeat-module-google_workspace.html

- module: google_workspace
  saml:
    enabled: true
    var.jwt_file: "/etc/filebeat/customer_json/xxx_googlews_auth.json"
    var.delegated_account: "gwm-service-account@gwm-xxx.iam.gserviceaccount.com"
    # var.jwt_file: credentials.json
    # var.delegated_account: admin@example.com
    # var.initial_interval: 24h
    # var.http_client_timeout: 60s
    # var.user_key: all
    # var.interval: 2h

This is the error i'm getting:

2023-07-14T13:37:53.483+0100    ERROR   [input.httpjson-cursor] v2/input.go:115 Error while processing http request: failed to execute http client.Do: server responded with$
  "error": {
    "code": 401,
    "message": "Access denied. You are not authorized to read activity records.",
    "errors": [
      {
        "message": "Access denied. You are not authorized to read activity records.",
        "domain": "global",
        "reason": "authError",
        "location": "Authorization",
        "locationType": "header"
      }
    ]
  }
}
        {"id": "97E5B91E284F6ECB", "input_source": "https://www.googleapis.com/admin/reports/v1/activity/users/all/applications/admin", "input_url": "https://www.googleapis.com/admin/reports/v1/activity/users/all/applications/admin"}

I've checked the client id, the service account name, the domain wide delegation and it is all as it should be.

Anyone else seen this issue?

Thanks
John

2

Hi @Nightingale_John,

Firstly, I'd recommend that you use our Workspace integration via Elastic Agent, as it's typically easier to configure and also includes several bug fixes and enhancements (broader event coverage and new dashboards) when compared to the Filebeat module.

With regards to the 401 error, it's often due to an issue with the domain wide delegation or a step missed when setting up the service account. Given all the steps required to set up the account on the Workspace side, it's easy to miss something. This blog post walks through the whole process step-by-step. I'd suggest going through it to ensure everything has been set up correctly, and if not, let me know and we can dig deeper.

3

Thanks Jamie,

Yes, I suspect something to do with the delegation too, but having gone through everything a number of times I'm stumped as to what is misconfigured.

I'll run through the blog post as you mentioned, thank you for that.

At the minute I'm using filebeat for legacy purposes, but intend to review Elastic Agent this week.

Thanks for the help, I'll respond here once I know more.

Best
John

1 Like
4

Hi Jamie,

I went through the blog post and followed as directed, I created a new project to test this out, and it comes out with the same result!

I'm currently using filebeat still, but intend to try it with elastic-agent this week.

Best
John

5

Hello! Progress... I got it working..

The delegated account needed to be set to the admin account that created the service account, rather than the service account that was created... is that expected?!

Best

6

Hey @Nightingale_John,

I came here looking for an answer as I'm having the same issue.

When you mention "The delegated account needed to be set to the admin account that created the service account" - is that the Google Workspace domain-wide delegation only? I can't find a client ID for my admin account so a bit lost. Does the Filebeat module still utilize the same Service Account json credential file or is that needed to be regenerated as well?

7

Hey!

So...

modules.d/google_workspace.yml

- module: google_workspace
  saml:
    enabled: true
    var.jwt_file: /path/to/credentials.json
    var.delegated_account: <adminaccount email address, e.g. ja-admin@domainname>

the service account email address is in the credentials.json file.

I didn't need to regenerate anything fortunately, I just needed to put the admin account email address of my admin account that I used to create the service account and the resulting JSON file.

Hopefully that makes sense.. but the var.delegated_account IS NOT the service account email address.

John

8

Got it! That is so.....not intuitive. Working after that.

1 Like
9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.