FILEBEAT - GOOGLE CLOUD MODULE - PUB/SUB

Elastic Stack Beats
,
1

Hi there:

Running Filebeat, 7.8.1 and using the Google Cloud Module, PUB/SUB, to pull out VPC Firewall Logs.

Getting an error (I believe JSON related) when Filebeat starts:

ERROR [google.pubsub] googlepubsub/input.go:127 pubsub: invalid character '\n' in string literal

The Filbeat config file looks like this:

    filebeat.inputs:
    - type: google-pubsub
      project_id: redacted
      topic: vpc-firewall-logs-topic
      subscription.name: filebeat-vpc-firewall-logs-sub
      credentials_file: ${path.config}/redacted.json
      subscription.create: false
      enabled: true
    filebeat.config.modules:
      path: ${path.config}/modules.d/*.yml
      reload.enabled: false
      document_type: json
      json.keys_under_root: true
      json.add_error_key: true
    output.logstash:
      hosts: ["redacted:5045"]
      ssl.enabled: true
      client_authentication: none

Any ideas?

Thanks!

Juan.

2

Could you please share debug logs of Filebeat (output of ./filebeat -e -d "*")?

3

Hi Noemi:

Thanks for your response.

Output attached here (several entries due to characters limit here):

2020-08-14T07:24:08.830+1000    INFO    instance/beat.go:647    Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]
2020-08-14T07:24:08.830+1000    DEBUG   [beat]  instance/beat.go:699    Beat metadata path: /var/lib/filebeat/meta.json
2020-08-14T07:24:08.830+1000    INFO    instance/beat.go:655    Beat ID: 34bdb566-4c8d-43fd-8938-9efbb12f69c7
2020-08-14T07:24:08.831+1000    DEBUG   [docker]        docker/client.go:48     Docker client will negotiate the API version on the first request.
2020-08-14T07:24:08.833+1000    DEBUG   [add_docker_metadata]   add_docker_metadata/add_docker_metadata.go:87   add_docker_metadata: docker environment not detected: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
2020-08-14T07:24:08.833+1000    DEBUG   [add_cloud_metadata]    add_cloud_metadata/providers.go:126     add_cloud_metadata: starting to fetch metadata, timeout=3s
2020-08-14T07:24:08.835+1000    DEBUG   [kubernetes]    add_kubernetes_metadata/kubernetes.go:138       Could not create kubernetes client using in_cluster config: unable to build kube config due to error: invalid configuration: no configuration has been provided      {"libbeat.processor": "add_kubernetes_metadata"}
2020-08-14T07:24:08.839+1000    DEBUG   [add_cloud_metadata]    add_cloud_metadata/providers.go:162     add_cloud_metadata: received disposition for digitalocean after 5.930874ms. result=[provider:digitalocean, error=failed with http status code 404, metadata={}]
2020-08-14T07:24:08.840+1000    DEBUG   [add_cloud_metadata]    add_cloud_metadata/providers.go:162     add_cloud_metadata: received disposition for openstack after 6.284636ms. result=[provider:openstack, error=failed with http status code 404, metadata={}]
2020-08-14T07:24:08.840+1000    DEBUG   [add_cloud_metadata]    add_cloud_metadata/providers.go:162     add_cloud_metadata: received disposition for az after 6.582425ms. result=[provider:az, error=failed with http status code 404, metadata={}]
2020-08-14T07:24:08.840+1000    DEBUG   [add_cloud_metadata]    add_cloud_metadata/providers.go:162     add_cloud_metadata: received disposition for aws after 6.870747ms. result=[provider:aws, error=failed with http status code 404, metadata={}]
2020-08-14T07:24:08.842+1000    DEBUG   [add_cloud_metadata]    add_cloud_metadata/providers.go:162     add_cloud_metadata: received disposition for gcp after 8.349071ms. result=[provider:gcp, error=<nil>, metadata={"availability_zone":"australia-southeast1-b","instance":{"id":"6552887070891700583","name":"jumphost"},"machine":{"type":"n1-standard-1"},"project":{"id":"redacted"},"provider":"gcp"}]
2020-08-14T07:24:08.842+1000    DEBUG   [add_cloud_metadata]    add_cloud_metadata/providers.go:129     add_cloud_metadata: fetchMetadata ran for 8.398806ms
2020-08-14T07:24:08.842+1000    INFO    [add_cloud_metadata]    add_cloud_metadata/add_cloud_metadata.go:93     add_cloud_metadata: hosting provider type detected as gcp, metadata={"availability_zone":"australia-southeast1-b","instance":{"id":"6552887070891700583","name":"jumphost"},"machine":{"type":"n1-standard-1"},"project":{"id":"redacted"},"provider":"gcp"}
2020-08-14T07:24:08.842+1000    DEBUG   [processors]    processors/processor.go:101     Generated new processors: add_host_metadata=[netinfo.enabled=[true], cache.ttl=[5m0s]], add_cloud_metadata={"availability_zone":"australia-southeast1-b","instance":{"id":"6552887070891700583","name":"jumphost"},"machine":{"type":"n1-standard-1"},"project":{"id":"redacted"},"provider":"gcp"}, add_docker_metadata=[match_fields=[] match_pids=[process.pid, process.ppid]], add_kubernetes_metadata
2020-08-14T07:24:08.842+1000    DEBUG   [seccomp]       seccomp/seccomp.go:117  Loading syscall filter  {"seccomp_filter": {"no_new_privs":true,"flag":"tsync","policy":{"default_action":"errno","syscalls":[{"names":["accept","accept4","access","arch_prctl","bind","brk","chmod","chown","clock_gettime","clone","close","connect","dup","dup2","epoll_create","epoll_create1","epoll_ctl","epoll_pwait","epoll_wait","exit","exit_group","fchdir","fchmod","fchmodat","fchown","fchownat","fcntl","fdatasync","flock","fstat","fstatfs","fsync","ftruncate","futex","getcwd","getdents","getdents64","geteuid","getgid","getpeername","getpid","getppid","getrandom","getrlimit","getrusage","getsockname","getsockopt","gettid","gettimeofday","getuid","inotify_add_watch","inotify_init1","inotify_rm_watch","ioctl","kill","listen","lseek","lstat","madvise","mincore","mkdirat","mmap","mprotect","munmap","nanosleep","newfstatat","open","openat","pipe","pipe2","poll","ppoll","pread64","pselect6","pwrite64","read","readlink","readlinkat","recvfrom","recvmmsg","recvmsg","rename","renameat","rt_sigaction","rt_sigprocmask","rt_sigreturn","sched_getaffinity","sched_yield","sendfile","sendmmsg","sendmsg","sendto","set_robust_list","setitimer","setsockopt","shutdown","sigaltstack","socket","splice","stat","statfs","sysinfo","tgkill","time","tkill","uname","unlink","unlinkat","wait4","waitid","write","writev"],"action":"allow"}]}}}
2020-08-14T07:24:08.842+1000    INFO    [seccomp]       seccomp/seccomp.go:124  Syscall filter successfully installed
2020-08-14T07:24:08.842+1000    INFO    [beat]  instance/beat.go:983    Beat info       {"system_info": {"beat": {"path": {"config": "/etc/filebeat", "data": "/var/lib/filebeat", "home": "/usr/share/filebeat", "logs": "/var/log/filebeat"}, "type": "filebeat", "uuid": "34bdb566-4c8d-43fd-8938-9efbb12f69c7"}}}
2020-08-14T07:24:08.842+1000    INFO    [beat]  instance/beat.go:992    Build info      {"system_info": {"build": {"commit": "94f7632be5d56a7928595da79f4b829ffe123744", "libbeat": "7.8.1", "time": "2020-07-21T15:12:45.000Z", "version": "7.8.1"}}}
2020-08-14T07:24:08.842+1000    INFO    [beat]  instance/beat.go:995    Go runtime info {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":1,"version":"go1.13.10"}}}
2020-08-14T07:24:08.843+1000    INFO    [beat]  instance/beat.go:999    Host info       {"system_info": {"host": {"architecture":"x86_64","boot_time":"2020-07-18T11:25:35+10:00","containerized":false,"name":"jumphost","ip":["127.0.0.1/8","::1/128","10.255.0.3/32","fe80::4001:aff:feff:3/64"],"kernel_version":"4.9.0-12-amd64","mac":["42:01:0a:ff:00:03"],"os":{"family":"debian","platform":"debian","name":"Debian GNU/Linux","version":"9 (stretch)","major":9,"minor":0,"patch":0,"codename":"stretch"},"timezone":"AEST","timezone_offset_sec":36000,"id":"68d8fd9b514d8e139dca1168ba4b3300"}}}
4 
2020-08-14T07:24:08.843+1000    INFO    [beat]  instance/beat.go:1028   Process info    {"system_info": {"process": {"capabilities": {"inheritable":null,"permitted":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"effective":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"bounding":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"ambient":null}, "cwd": "/home/jromero", "exe": "/usr/share/filebeat/bin/filebeat", "name": "filebeat", "pid": 15072, "ppid": 15042, "seccomp": {"mode":"filter"}, "start_time": "2020-08-14T07:24:07.870+1000"}}}
2020-08-14T07:24:08.843+1000    INFO    instance/beat.go:310    Setup Beat: filebeat; Version: 7.8.1
2020-08-14T07:24:08.843+1000    DEBUG   [beat]  instance/beat.go:336    Initializing output plugins
2020-08-14T07:24:08.845+1000    DEBUG   [publisher]     pipeline/consumer.go:137        start pipeline event consumer
2020-08-14T07:24:08.845+1000    INFO    [publisher]     pipeline/module.go:113  Beat name: jumphost
2020-08-14T07:24:08.847+1000    WARN    beater/filebeat.go:156  Filebeat is unable to load the Ingest Node pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the Ingest Node pipelines or are using Logstash pipelines, you can ignore this warning.
2020-08-14T07:24:08.847+1000    INFO    instance/beat.go:463    filebeat start running.
2020-08-14T07:24:08.847+1000    DEBUG   [test]  registrar/migrate.go:159        isFile(/var/lib/filebeat/registry) -> false
2020-08-14T07:24:08.847+1000    DEBUG   [test]  registrar/migrate.go:159        isFile() -> false
2020-08-14T07:24:08.848+1000    DEBUG   [test]  registrar/migrate.go:152        isDir(/var/lib/filebeat/registry/filebeat) -> true
2020-08-14T07:24:08.848+1000    DEBUG   [test]  registrar/migrate.go:159        isFile(/var/lib/filebeat/registry/filebeat/meta.json) -> true
2020-08-14T07:24:08.848+1000    DEBUG   [registrar]     registrar/migrate.go:51 Registry type '0' found
2020-08-14T07:24:08.848+1000    DEBUG   [registrar]     registrar/registrar.go:125      Registry file set to: /var/lib/filebeat/registry/filebeat/data.json
2020-08-14T07:24:08.848+1000    WARN    beater/filebeat.go:339  Filebeat is unable to load the Ingest Node pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the Ingest Node pipelines or are using Logstash pipelines, you can ignore this warning.
2020-08-14T07:24:08.848+1000    INFO    registrar/registrar.go:145      Loading registrar data from /var/lib/filebeat/registry/filebeat/data.json
2020-08-14T07:24:08.848+1000    INFO    registrar/registrar.go:152      States Loaded from registrar: 0
2020-08-14T07:24:08.848+1000    INFO    [crawler]       beater/crawler.go:71    Loading Inputs: 1
2020-08-14T07:24:08.848+1000    INFO    [google.pubsub] googlepubsub/input.go:112       Initialized Google Pub/Sub input.       {"pubsub_project": "redacted", "pubsub_topic": "vpc-firewall-logs-topic", "pubsub_subscription": {"Name":"filebeat-vpc-firewall-logs-sub","NumGoroutines":1,"MaxOutstandingMessages":1000,"Create":false}}
2020-08-14T07:24:08.848+1000    INFO    [crawler]       beater/crawler.go:141   Starting input (ID: 11448794657429154599)
2020-08-14T07:24:08.848+1000    DEBUG   [cfgfile]       cfgfile/reload.go:132   Checking module configs from: /etc/filebeat/modules.d/*.yml
2020-08-14T07:24:08.849+1000    DEBUG   [cfgfile]       cfgfile/cfgfile.go:193  Load config from file: /etc/filebeat/modules.d/googlecloud.yml
2020-08-14T07:24:08.849+1000    DEBUG   [cfgfile]       cfgfile/reload.go:146   Number of module configs found: 1
2020-08-14T07:24:08.857+1000    DEBUG   [processor.javascript]  javascript/session.go:162       Registered params with processor
2020-08-14T07:24:08.858+1000    DEBUG   [processors]    processors/processor.go:101     Generated new processors: script=[type=javascript, id=, sources=/usr/share/filebeat/module/googlecloud/firewall/config/pipeline.js], add_fields={"ecs":{"version":"1.5.0"}}
2020-08-14T07:24:08.858+1000    INFO    [google.pubsub] googlepubsub/input.go:112       Initialized Google Pub/Sub input.       {"pubsub_project": "redacted", "pubsub_topic": "projects/redacted/topics/vpc-firewall-logs-topic", "pubsub_subscription": {"Name":"projects/redacted/subscriptions/filebeat-vpc-firewall-logs-sub","NumGoroutines":1,"MaxOutstandingMessages":1000,"Create":true}}
2020-08-14T07:24:08.858+1000    INFO    [crawler]       beater/crawler.go:108   Loading and starting Inputs completed. Enabled inputs: 1
2020-08-14T07:24:08.859+1000    INFO    [monitoring]    log/log.go:118  Starting metrics logging every 30s
2020-08-14T07:24:08.859+1000    DEBUG   [registrar]     registrar/registrar.go:278      Starting Registrar
2020-08-14T07:24:08.859+1000    INFO    [google.pubsub] googlepubsub/input.go:122       Pub/Sub input worker has started.       {"pubsub_project": "redacted", "pubsub_topic": "vpc-firewall-logs-topic", "pubsub_subscription": {"Name":"filebeat-vpc-firewall-logs-sub","NumGoroutines":1,"MaxOutstandingMessages":1000,"Create":false}}
2020-08-14T07:24:08.859+1000    ERROR   [google.pubsub] googlepubsub/input.go:127       pubsub: invalid character '\n' in string literal        {"pubsub_project": "redacted", "pubsub_topic": "vpc-firewall-logs-topic", "pubsub_subscription": {"Name":"filebeat-vpc-firewall-logs-sub","NumGoroutines":1,"MaxOutstandingMessages":1000,"Create":false}}
2020-08-14T07:24:08.859+1000    INFO    [google.pubsub] googlepubsub/input.go:128       Pub/Sub input worker has stopped.       {"pubsub_project": "redacted", "pubsub_topic": "vpc-firewall-logs-topic", "pubsub_subscription": {"Name":"filebeat-vpc-firewall-logs-sub","NumGoroutines":1,"MaxOutstandingMessages":1000,"Create":false}}
2020-08-14T07:24:08.859+1000    INFO    cfgfile/reload.go:164   Config reloader started
2020-08-14T07:24:08.860+1000    DEBUG   [cfgfile]       cfgfile/reload.go:194   Scan for new config files
2020-08-14T07:24:08.860+1000    DEBUG   [cfgfile]       cfgfile/cfgfile.go:193  Load config from file: /etc/filebeat/modules.d/googlecloud.yml
2020-08-14T07:24:08.861+1000    DEBUG   [cfgfile]       cfgfile/reload.go:213   Number of module configs found: 1
2020-08-14T07:24:08.862+1000    DEBUG   [reload]        cfgfile/list.go:62      Starting reload procedure, current runners: 0
2020-08-14T07:24:08.862+1000    DEBUG   [reload]        cfgfile/list.go:80      Start list: 1, Stop list: 0
2020-08-14T07:24:08.870+1000    DEBUG   [processor.javascript]  javascript/session.go:162       Registered params with processor
5 
2020-08-14T07:24:08.870+1000    DEBUG   [processors]    processors/processor.go:101     Generated new processors: script=[type=javascript, id=, sources=/usr/share/filebeat/module/googlecloud/firewall/config/pipeline.js], add_fields={"ecs":{"version":"1.5.0"}}
2020-08-14T07:24:08.870+1000    INFO    [google.pubsub] googlepubsub/input.go:112       Initialized Google Pub/Sub input.       {"pubsub_project": "redacted", "pubsub_topic": "projects/redacted/topics/vpc-firewall-logs-topic", "pubsub_subscription": {"Name":"projects/redacted/subscriptions/filebeat-vpc-firewall-logs-sub","NumGoroutines":1,"MaxOutstandingMessages":1000,"Create":true}}
2020-08-14T07:24:08.871+1000    DEBUG   [reload]        cfgfile/list.go:101     Starting runner: googlecloud (firewall)
2020-08-14T07:24:08.871+1000    INFO    cfgfile/reload.go:224   Loading of config files completed.
2020-08-14T07:24:08.871+1000    INFO    [google.pubsub] googlepubsub/input.go:122       Pub/Sub input worker has started.       {"pubsub_project": "redacted", "pubsub_topic": "projects/redacted/topics/vpc-firewall-logs-topic", "pubsub_subscription": {"Name":"projects/redacted/subscriptions/filebeat-vpc-firewall-logs-sub","NumGoroutines":1,"MaxOutstandingMessages":1000,"Create":true}}
2020-08-14T07:24:08.871+1000    ERROR   [google.pubsub] googlepubsub/input.go:127       pubsub: invalid character '\n' in string literal        {"pubsub_project": "redacted", "pubsub_topic": "projects/redacted/topics/vpc-firewall-logs-topic", "pubsub_subscription": {"Name":"projects/redacted/subscriptions/filebeat-vpc-firewall-logs-sub","NumGoroutines":1,"MaxOutstandingMessages":1000,"Create":true}}
2020-08-14T07:24:08.871+1000    INFO    [google.pubsub] googlepubsub/input.go:128       Pub/Sub input worker has stopped.       {"pubsub_project": "redacted", "pubsub_topic": "projects/redacted/topics/vpc-firewall-logs-topic", "pubsub_subscription": {"Name":"projects/redacted/subscriptions/filebeat-vpc-firewall-logs-sub","NumGoroutines":1,"MaxOutstandingMessages":1000,"Create":true}}
2020-08-14T07:24:18.859+1000    DEBUG   [input] input/input.go:141      Run input
2020-08-14T07:24:18.871+1000    DEBUG   [input] input/input.go:141      Run input
2020-08-14T07:24:28.859+1000    DEBUG   [input] input/input.go:141      Run input
2020-08-14T07:24:28.871+1000    DEBUG   [input] input/input.go:141      Run input
2020-08-14T07:24:38.861+1000    INFO    [monitoring]    log/log.go:145  Non-zero metrics in the last 30s        {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":20,"time":{"ms":28}},"total":{"ticks":120,"time":{"ms":132},"value":120},"user":{"ticks":100,"time":{"ms":104}}},"handles":{"limit":{"hard":1048576,"soft":1024},"open":9},"info":{"ephemeral_id":"fa62c3cb-3eba-4879-b621-b4d31d65b32e","uptime":{"ms":30088}},"memstats":{"gc_next":13918880,"memory_alloc":7348960,"memory_total":20341384,"rss":56692736},"runtime":{"goroutines":27}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0},"reloads":1,"scans":1},"output":{"type":"logstash"},"pipeline":{"clients":2,"events":{"active":0}}},"registrar":{"states":{"current":0}},"system":{"cpu":{"cores":1},"load":{"1":0,"15":0.21,"5":0.08,"norm":{"1":0,"15":0.21,"5":0.08}}}}}}
2020-08-14T07:24:38.861+1000    DEBUG   [input] input/input.go:141      Run input
2020-08-14T07:24:38.871+1000    DEBUG   [input] input/input.go:141      Run input
^C2020-08-14T07:24:39.425+1000  DEBUG   [service]       service/service.go:54   Received sigterm/sigint, stopping
2020-08-14T07:24:39.425+1000    INFO    beater/filebeat.go:456  Stopping filebeat
2020-08-14T07:24:39.425+1000    INFO    beater/crawler.go:148   Stopping Crawler
2020-08-14T07:24:39.425+1000    INFO    beater/crawler.go:158   Stopping 1 inputs
2020-08-14T07:24:39.425+1000    INFO    cfgfile/reload.go:227   Dynamic config reloader stopped
2020-08-14T07:24:39.425+1000    INFO    [reload]        cfgfile/list.go:118     Stopping 1 runners ...
2020-08-14T07:24:39.425+1000    DEBUG   [reload]        cfgfile/list.go:129     Stopping runner: googlecloud (firewall)
2020-08-14T07:24:39.425+1000    INFO    [crawler]       beater/crawler.go:163   Stopping input: 11448794657429154599
2020-08-14T07:24:39.425+1000    INFO    input/input.go:138      input ticker stopped
2020-08-14T07:24:39.425+1000    DEBUG   [reload]        cfgfile/list.go:131     Stopped runner: googlecloud (firewall)
2020-08-14T07:24:39.425+1000    INFO    input/input.go:138      input ticker stopped
2020-08-14T07:24:39.425+1000    INFO    beater/crawler.go:178   Crawler stopped
2020-08-14T07:24:39.425+1000    INFO    registrar/registrar.go:367      Stopping Registrar
2020-08-14T07:24:39.425+1000    INFO    registrar/registrar.go:293      Ending Registrar
2020-08-14T07:24:39.425+1000    DEBUG   [registrar]     registrar/registrar.go:411      Write registry file: /var/lib/filebeat/registry/filebeat/data.json (0)
2020-08-14T07:24:39.426+1000    DEBUG   [publisher]     pipeline/client.go:162  client: closing acker
2020-08-14T07:24:39.426+1000    DEBUG   [publisher]     pipeline/client.go:173  client: done closing acker
2020-08-14T07:24:39.426+1000    DEBUG   [publisher]     pipeline/client.go:176  client: cancelled 0 events
2020-08-14T07:24:39.426+1000    DEBUG   [publisher]     pipeline/client.go:147  client: wait for acker to finish
2020-08-14T07:24:39.426+1000    DEBUG   [publisher]     pipeline/client.go:149  client: acker shut down
2020-08-14T07:24:39.426+1000    DEBUG   [publisher]     pipeline/client.go:162  client: closing acker
2020-08-14T07:24:39.426+1000    DEBUG   [publisher]     pipeline/client.go:173  client: done closing acker
2020-08-14T07:24:39.426+1000    DEBUG   [publisher]     pipeline/client.go:176  client: cancelled 0 events
2020-08-14T07:24:39.426+1000    DEBUG   [publisher]     pipeline/client.go:147  client: wait for acker to finish
2020-08-14T07:24:39.426+1000    DEBUG   [publisher]     pipeline/client.go:149  client: acker shut down
2020-08-14T07:24:39.430+1000    DEBUG   [registrar]     registrar/registrar.go:404      Registry file updated. 0 states written.
2020-08-14T07:24:39.432+1000    INFO    [monitoring]    log/log.go:153  Total non-zero metrics  {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":20,"time":{"ms":28}},"total":{"ticks":120,"time":{"ms":136},"value":120},"user":{"ticks":100,"time":{"ms":108}}},"handles":{"limit":{"hard":1048576,"soft":1024},"open":9},"info":{"ephemeral_id":"fa62c3cb-3eba-4879-b621-b4d31d65b32e","uptime":{"ms":30659}},"memstats":{"gc_next":13918880,"memory_alloc":7777224,"memory_total":20769648,"rss":56692736},"runtime":{"goroutines":14}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0},"reloads":1,"scans":1},"output":{"type":"logstash"},"pipeline":{"clients":0,"events":{"active":0}}},"registrar":{"states":{"current":0},"writes":{"success":1,"total":1}},"system":{"cpu":{"cores":1},"load":{"1":0,"15":0.2,"5":0.08,"norm":{"1":0,"15":0.2,"5":0.08}}}}}}
2020-08-14T07:24:39.432+1000    INFO    [monitoring]    log/log.go:154  Uptime: 30.660762122s
2020-08-14T07:24:39.432+1000    INFO    [monitoring]    log/log.go:131  Stopping metrics logging.
2020-08-14T07:24:39.432+1000    INFO    instance/beat.go:469    filebeat stopped.
6

Are you intentionally running two instances of the pub/sub input? You have one in your filebeat.inputs in filebeat.yml and you also have the googlecloud module running from modules.d/.

I suspect the error stems from the creation of the pub/sub client within the input. While not 100% clear I think it's happening while reading your credentails JSON file. Can you validate the file with jq or some other tool to check that the file contains valid JSON (e.g. jq . redacted.json).

7

Hi Andrew:
Thx for your help.
The issue seemed to be indeed loading from inputs and modules.d
Removing it from inputs solved this :slight_smile:
Thanks again
Juan.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.