Postgresql Module Missing Field Errors

[kwojcicki]

Using PostgreSQL with dockerimage: library/postgres:9.5-alpine my metricbeat container (5.5.0) spits out the following errors:

2017/09/11 16:41:56.911310 schema.go:38: ERR Error on field 'query_start': Error converting param to time.Time: query_start. Original:
2017/09/11 16:41:56.911327 schema.go:38: ERR Error on field 'waiting': Key waiting not found

My metricbeat.yml looks like the following (omitted sending to ES etc)

• module: postgresql
  name: postgresql
  enabled: true
  period: 10s
  metricsets: ["database", "bgwriter", "activity"]
  hosts: ["postgres://postgres:5432?sslmode=disable"]
  username: postgres
  password: postgres

[kwojcicki]

Was actually a problem due to one of the microservices not connecting to postgresql correctly and leaving an empty row in pg_stat_activity;

[andrewkroh]

So if there has never been any activity does that mean a user would encounter these error messages with this module?

[kwojcicki]

Yup exactly if you run a similar query to what metric beat does (select usename, query_start from pg_stat_activity;) you will get something like the following (i am using postgres 9.5 and metricbeat 9.5 so column names can vary but general idea should hold)

postgres=# select usename, query_start from pg_stat_activity;
   usename    |          query_start          
--------------+-------------------------------
 service1     | 2017-09-12 14:41:55.176988+00
 service1     | 2017-09-12 14:41:49.329056+00
 service3     |
 service2     | 2017-09-12 15:07:55.134885+00
 service2     | 2017-09-12 14:41:51.422839+00
 service2     | 2017-09-12 14:41:51.42319+00

And as you can see one of the service3 rows has no query_start since only a connection has been established no query has actually be run yet. I think metricbeat acted as expected but perhaps making that field optional could be a good idea?