Functionbeat GCP pub/sub requiring storage entry point when only pub/sub is enabled

Elastic Stack Beats
,
1

Deploying functionbeat on GCP with entry point RunPubSub returns an error complaining about storage entry point, which I am not using.

I get the following error upon deploy

Deployment failure:
Build failed: # functions.local/app/main
src/functions.local/app/main/main.go:43:21: undefined: storage.RunPubSub; Error ID: 2f5e35a0

The problem is ,as I noticed it, Functionbeat is requiring storage entry point. I am NOT using the gcp storage functionality, I am using pub/sub functionality with RunPubSub. Why is it requiring the storage entry point? I am not able to provide multiple entry points to my knowledge.

  1. Package functionbeat using LINUX X86_64 7.13.4 up using the following command
./functionbeat  export function pubsub -c functionbeat-file.yml
  1. Go into GCP function and go through creating a cloud function steps, upload the packaed zip,
  2. add entry point "RunPubSub"
  3. Select Golang version 1.16 in order to avoid another bug (functionbeat 7.12 can't deploy -- unknown field 'VerifyConnection' in struct literal of type tls.Config · Issue #24925 · elastic/beats · GitHub). <<< Selecting 1.13 will yield this error. Keep in mind the runtime of the function is GoLang 1.13 (could this be the problem?)
 ./functionbeat export function pubsub |grep runtime
    runtime: go113

My config file

functionbeat.provider.gcp.location_id: "us-central1"

functionbeat.provider.gcp.project_id: "some-project"

functionbeat.provider.gcp.storage_name: "some-bucket"

functionbeat.provider.gcp.functions:
  - name: pubsub
    enabled: true
    type: pubsub

    description: "Google Cloud Function for Pub/Sub"
    memory_size: 512MB
    timeout: 60s
    service_account_email: some-email

    trigger:
      resource: "some-topic"
processors:
- add_cloud_metadata:
    providers: ["gcp"]
- add_fields:
    target: field
    fields:
      sourcetype: test
      service: logging
- drop_event:
    when:
      not:
       regexp:
         field.sourcetype: ".*"
logging.level: info
logging.json: true
output.logstash:
  hosts: ["some-host"]
  pipelining: 0
  max_retries: 0
  ttl: 30s
setup.ilm.enabled: false
setup.template.enabled: false
path:
  config: ./
  home: ./

Relating: Functionbeat missing package io/fs - #10 by Edoardo_Tenani

2

I wonder even if I manage to get this to run if I will run into this

GCP/FunctionBeat Errors [cannot find matching process for pid=1, name version already used] · Issue #25596 · elastic/beats · GitHub 1

3

Keep in mind the runtime of the function is GoLang 1.13 (could this be the problem?)
recompiled the functionbeat binary as goLang 1.16 and repackaged, same error.

4

You should avoid getting functionbeat to run on cloud function. It doesn't work and it will never reach a proper release. Edoardo told me about using a GCP dataflow template and it's working fine so far. The downside is the cost tho.

5

Thanks, - just surprised that they included gcp support for so long until 7.15

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.