Unable to get visualize data from postgres fusing metricbeat

Elastic Stack Beats
1

Hi,

i am using elk 5.6, i have installed elasticsearch and kibana successfully. i have configured metric beat , i am able to get all cpu metrics but am not gettign any metrics regrding postgres.

while the metric beat log says it is sending data to elastisearch. but when i try to visualize am gettign null value for all the postgres related data. kindly help to resolve the same.
also is there a standard dashboard for postgres.

2

Can you share your metricbeat config and the logs?

3

hi rufin pfb the error log i am getting from metric beat.
2017/11/16 05:41:27.217642 wrapper.go:201: DBG Stopped metricSetWrapper[module=system, name=diskio, host=]
2017/11/16 05:41:27.217650 wrapper.go:201: DBG Stopped metricSetWrapper[module=system, name=filesystem, host=]
2017/11/16 05:41:27.217666 wrapper.go:201: DBG Stopped metricSetWrapper[module=system, name=core, host=]
2017/11/16 05:41:27.217631 wrapper.go:201: DBG Stopped metricSetWrapper[module=system, name=load, host=]
2017/11/16 05:41:27.217672 wrapper.go:201: DBG Stopped metricSetWrapper[module=system, name=cpu, host=]
2017/11/16 05:41:27.217675 wrapper.go:201: DBG Stopped metricSetWrapper[module=postgresql, name=activity, host=172.21.197.195:5444]
2017/11/16 05:41:27.217982 metrics.go:51: INFO Total non-zero values: fetches.postgresql-activity.events=2 fetches.postgresql-activity.failures=2 fetches.postgresql-bgwriter.events=4 fetches.postgresql-bgwriter.failures=4 fetches.postgresql-database.events=2 fetches.postgresql-database.failures=2 fetches.system-core.events=12 fetches.system-core.success=2 fetches.system-cpu.events=2 fetches.system-cpu.success=2 fetches.system-diskio.events=16 fetches.system-diskio.success=2 fetches.system-filesystem.events=84 fetches.system-filesystem.success=2 fetches.system-fsstat.events=2 fetches.system-fsstat.success=2 fetches.system-load.events=2 fetches.system-load.success=2 fetches.system-memory.events=2 fetches.system-memory.success=2 fetches.system-network.events=10 fetches.system-network.success=2 fetches.system-process.events=1120 fetches.system-process.success=2 libbeat.es.call_count.PublishEvents=26 libbeat.es.publish.read_bytes=12765 libbeat.es.publish.write_bytes=690304 libbeat.es.published_and_acked_events=1258 libbeat.publisher.messages_in_worker_queues=1258 libbeat.publisher.published_events=1258
2017/11/16 05:41:27.218001 metrics.go:52: INFO Uptime: 15.258655504s
2017/11/16 05:41:27.218005 beat.go:237: INFO metricbeat stopped.
2017/11/16 05:41:27.217642 wrapper.go:201: DBG Stopped metricSetWrapper[module=system, name=diskio, host=]
2017/11/16 05:41:27.217650 wrapper.go:201: DBG Stopped metricSetWrapper[module=system, name=filesystem, host=]
2017/11/16 05:41:27.217666 wrapper.go:201: DBG Stopped metricSetWrapper[module=system, name=core, host=]
2017/11/16 05:41:27.217631 wrapper.go:201: DBG Stopped metricSetWrapper[module=system, name=load, host=]
2017/11/16 05:41:27.217672 wrapper.go:201: DBG Stopped metricSetWrapper[module=system, name=cpu, host=]
2017/11/16 05:41:27.217675 wrapper.go:201: DBG Stopped metricSetWrapper[module=postgresql, name=activity, host=172.21.197.195:5444]
2017/11/16 05:41:27.217982 metrics.go:51: INFO Total non-zero values: fetches.postgresql-activity.events=2 fetches.postgresql-activity.failures=2 fetches.postgresql-bgwriter.events=4 fetches.postgresql-bgwriter.failures=4 fetches.postgresql-database.events=2 fetches.postgresql-database.failures=2 fetches.system-core.events=12 fetches.system-core.success=2 fetches.system-cpu.events=2 fetches.system-cpu.success=2 fetches.system-diskio.events=16 fetches.system-diskio.success=2 fetches.system-filesystem.events=84 fetches.system-filesystem.success=2 fetches.system-fsstat.events=2 fetches.system-fsstat.success=2 fetches.system-load.events=2 fetches.system-load.success=2 fetches.system-memory.events=2 fetches.system-memory.success=2 fetches.system-network.events=10 fetches.system-network.success=2 fetches.system-process.events=1120 fetches.system-process.success=2 libbeat.es.call_count.PublishEvents=26 libbeat.es.publish.read_bytes=12765 libbeat.es.publish.write_bytes=690304 libbeat.es.published_and_acked_events=1258 libbeat.publisher.messages_in_worker_queues=1258 libbeat.publisher.published_events=1258
2017/11/16 05:41:27.218001 metrics.go:52: INFO Uptime: 15.258655504s
2017/11/16 05:41:27.218005 beat.go:237: INFO metricbeat stopped.

4

pfb the metricbeat.yml


#==========================  Modules configuration ============================
metricbeat.modules:
  -
    module: system
    metricsets:
      - cpu
      - load
      - core
      - diskio
      - filesystem
      - fsstat
      - memory
      - network
      - process
    period: 10s
    processes:
      - .*
  -
    module: postgresql
    metricsets:
      - database
      - bgwriter
      - bgwriter
      - activity
    hosts:
      - "postgres://172.21.197.195:5444/SANTEDB"
    period: 10s

output.elasticsearch:
  hosts:
    - "172.21.197.152:9200"
#  password: changeme
#  username: elastic

logging.level: debug
5

Any chance you could post a little bit a longer log and from the beginning when metricbeat starts up and then wait for about 60s? For a test run could you disable the system module?

For the log best paste it into a gist and link it here.

6

Hi Ruflin

this is the error message i am getting, post commenting out the system module and trying only for the pgsql.

"error": "QueryStats: pq: unexpected error: "setting PGLOCALEDIR not supported"",
** "metricset":**

m2
m2.png1360×768 85.4 KB
m1
m1.png1360×768 117 KB
m2
m2.png1360×768 85.4 KB

the PGLOCALEDIR dir is already set, let me know if metricbeat expects any specific locale. the below locale are present in my system. thanks in advance
zh_TW
pt_BR
ta
ja
nb
zh_CN
it
de
id
es
fr
ro
cs
ru
sv
ko
tr
pl

7

From the go client library we use: pq package - github.com/lib/pq - Go Packages

Most environment variables as specified at PostgreSQL: Documentation: 16: 34.15. Environment Variables supported by libpq are also supported by pq. If any of the environment variables not supported by pq are set, pq will panic during connection establishment. Environment variables have a lower precedence than explicitly provided connection parameters.

Can you try unset the env variable?

In the future please paste text instead of screenshots.

8

HI Ruflin,
post unsetting the PGLOCALEDIR env variable i am able to proceed with metricbeat. thanks fot the guidance.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.