Filebeat Not Using Provided Credentials File w/ GCP Module

Elastic Stack Beats
1

Version: 7.12.1

I am getting the following when trying to enable the GCP module, using audit, firewall, and vpcflow filesets using var.credentials_file: /etc/filebeat/filebeat/creds.json :

Exiting: Failed to start crawler: starting input failed: Error while initializing input: no authentication credentials were configured or detected (credentials_file, credentials_json, and application default credentials (ADC)) accessing config

...even though I've checked the path (and permissions) for the credentials file, and it is correct. I've tried modifying path.config, etc as well, with no luck. If I explicitly set the credentials_file value in the modules.d/gcp.yml, it works, so it seems like it is not getting picked up somewhere.

2

Hello @cappy .

I am a bit unsure what you mean, all module configuration needs to happen in modules.d/modulename.yml

So if you are inserting it into gcp.yml then that should be correct, are you trying to add it somewhere else?

3

I see. Thank you for that. I must have misread, and was attempting to add the configuration to filebeat.yml under filebeat.modules, but it seems as though this is not the intended way to configure modules. It seems like this would be a centralized way to handle this, so I'm a bit surprised you need to configure them otherwise. Are there any plans to make this more centralized in the future?

For example, I added the following to filebeat.yml, and it seemed to work, but maybe it only worked because of coincidence?

- module: fortinet
  firewall:
    enabled: true
    var.input: udp
    var.syslog_host: 0.0.0.0
    var.syslog_port: 514
    input:
      fields_under_root: true
4

The reasons for it working is as you might see a bit later in filebeat.yml, we include all files under modules.d that ends with .yml so that they are combined in the end.

The reason this is split up, is because we have hundreds of integrations, so we would need to split them up somehow.

In terms of more automated and centralized, we do have our new Elastic Agent and Fleet in beta, which I think is what you are referring to. This will centralize all the beat management, config management etc in Kibana.
For some more information on this:

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.