Metricset 'postgresql/bgwriter' is not registered, module not found

Elastic Stack Beats
1

I have configure metricbeat to send postgresql metric but get this error when starting filebeat:

2016/09/20 13:54:29.102566 metricbeat.go:26: INFO Register [ModuleFactory:[], MetricSetFactory:[apache/status, mongodb/status, mysql/status, nginx/stubstatus, redis/info, redis/keyspace, system/core, system/cpu, system/diskio, system/filesystem, system/fsstat, system/load, system/memory, system/network, system/process, zookeeper/mntr]]
2016/09/20 13:54:29.201255 beat.go:284: CRIT Exiting: 1 error: metricset 'postgresql/bgwriter' is not registered, module not found
Exiting: 1 error: metricset 'postgresql/bgwriter' is not registered, module not found

config

- module: postgresql
  metricsets:
    # Stats about every PostgreSQL database
#    - database

    # Stats about the background writer process's activity
    - bgwriter

    # Stats about every PostgreSQL process
#    - activity

#  enabled: true
  period: 60s

  # The host must be passed as PostgreSQL DSN. Example:
  # postgres://pqgotest:password@localhost:5432?sslmode=disable
  # The available parameters are documented here:
  # https://godoc.org/github.com/lib/pq#hdr-Connection_String_Parameters
  hosts: ["postgres://postgres@localhost:5432"]
2

Which version of metricbeat are you using?

3

5.0-alpha5

4

The postgresql module wasn't part of Metricbeat 5.0-alpha5. It will be part of the next release.

5

If you like to test the postgres module you can find here the snapshot builds: https://beats-nightlies.s3.amazonaws.com/index.html?prefix=metricbeat/

6

I just see there 6.0-alpha1, no 5.0-alpha6. Is 6.0-alpha1 the right one and compatible with ES 5.0-alpha5?

7

6.0-alpha1 is currently the right one. We had to do some branch renaming in preparation for beta1. You are fine with this snapshot for the moment to test postgres.

8

I installed the 6.0-alpha1 but there is no import_dashboards script on /usr/share/metricbeat/kibana

I am using the old dsahboard from old 5.0-alpha5 ... is there a dashboard for postgresql?

9

Also getting this error on documents:

error QueryStats: pq: unexpected error: "setting PGSERVICEFILE not supported"

10
  • The import_dashboards script should be under metricbeat/scripts/*
  • There are currently no pre-built dashboards for postgresql. We are more then happy to get external contributions here :wink:

Where exactly do you get this error?

11

I get the error in Kibana while exploring the documents created by the postgres module

12

Could you please provide the JSON document as an example. What version of Postgres are you running?

13

I am running postgresql 9.2.5 on Centos 6. We cant upgrade it. We have same version on thousends of appliances.

JSON document:

{
  "_index": "metricbeat-2016.09.23",
  "_type": "metricsets",
  "_id": "AVdZCXMREu-Rb_NMhp9v",
  "_score": null,
  "_source": {
    "@timestamp": "2016-09-23T19:52:55.122Z",
    "beat": {
      "hostname": "rc02",
      "name": "rc02"
    },
    "error": "QueryStats: pq: unexpected error: \"setting PGSERVICEFILE not supported\"",
    "fields": {
      "asset_tag": "822-101-50240"
    },
    "metricset": {
      "host": "postgres://postgres@localhost:5432",
      "module": "postgresql",
      "name": "database",
      "rtt": 83
    },
    "postgresql": {
      "database": {}
    },
    "type": "metricsets"
  },
  "fields": {
    "@timestamp": [
      1474660375122
    ],
    "system.network.total.bytes": [
      0
    ]
  },
  "highlight": {
    "metricset.module": [
      "@kibana-highlighted-field@postgresql@/kibana-highlighted-field@"
    ]
  },
  "sort": [
    1474660375122
  ]
}
14

Does your environment have PGSERVICEFILE set?

From: pq package - github.com/lib/pq - Go Packages

Most environment variables as specified at PostgreSQL: Documentation: 16: 34.15. Environment Variables supported by libpq are also supported by pq. If any of the environment variables not supported by pq are set, pq will panic during connection establishment. Environment variables have a lower precedence than explicitly provided connection parameters.

15

it is set:

[root@rc02 ~]# echo $PGSERVICEFILE
/etc/sysconfig/pgsql/pg_service.conf

16

any update on this?

17

@agonzalez As we use the https://godoc.org/github.com/lib/pq as our library, the only workaround seems to be to remove the PGSERVICEFILE environment variable.

18

I edited /etc/init.d/metricbeat and added:

unset -v PGSERVICEFILE
unset -v PGSYSCONFDIR

Now i am getting other message error:

QueryStats: pq: unsupported sslmode "allow"; only "require" (default), "verify-full", "verify-ca", and "disable" supported

Any idea?

19

That's probably caused by having PGSSLMODE=allow in your environment. I'd unset that one as well. It appears the the Golang lib/pq doesn't agree with the C libpq on what the options are for this setting.

Ref: conn.go
libpq docs

20

thanks! that worked i am getting now the metrics.

Shouldnt you include by default in the metricbeat init.d the commands to unset the vars?

Is there an easy way to extend it to collect other info from postgresql? like runing a QUERY and sending the result to elk?