Unable to get visualize data from postgres fusing metricbeat

Elastic Stack Beats
#1

Hi,

i am using elk 5.6, i have installed elasticsearch and kibana successfully. i have configured metric beat , i am able to get all cpu metrics but am not gettign any metrics regrding postgres.

while the metric beat log says it is sending data to elastisearch. but when i try to visualize am gettign null value for all the postgres related data. kindly help to resolve the same.
also is there a standard dashboard for postgres.

#2

Can you share your metricbeat config and the logs?

#3

hi rufin pfb the error log i am getting from metric beat.
2017/11/16 05:41:27.217642 wrapper.go:201: DBG Stopped metricSetWrapper[module=system, name=diskio, host=]
2017/11/16 05:41:27.217650 wrapper.go:201: DBG Stopped metricSetWrapper[module=system, name=filesystem, host=]
2017/11/16 05:41:27.217666 wrapper.go:201: DBG Stopped metricSetWrapper[module=system, name=core, host=]
2017/11/16 05:41:27.217631 wrapper.go:201: DBG Stopped metricSetWrapper[module=system, name=load, host=]
2017/11/16 05:41:27.217672 wrapper.go:201: DBG Stopped metricSetWrapper[module=system, name=cpu, host=]
2017/11/16 05:41:27.217675 wrapper.go:201: DBG Stopped metricSetWrapper[module=postgresql, name=activity, host=172.21.197.195:5444]
2017/11/16 05:41:27.217982 metrics.go:51: INFO Total non-zero values: fetches.postgresql-activity.events=2 fetches.postgresql-activity.failures=2 fetches.postgresql-bgwriter.events=4 fetches.postgresql-bgwriter.failures=4 fetches.postgresql-database.events=2 fetches.postgresql-database.failures=2 fetches.system-core.events=12 fetches.system-core.success=2 fetches.system-cpu.events=2 fetches.system-cpu.success=2 fetches.system-diskio.events=16 fetches.system-diskio.success=2 fetches.system-filesystem.events=84 fetches.system-filesystem.success=2 fetches.system-fsstat.events=2 fetches.system-fsstat.success=2 fetches.system-load.events=2 fetches.system-load.success=2 fetches.system-memory.events=2 fetches.system-memory.success=2 fetches.system-network.events=10 fetches.system-network.success=2 fetches.system-process.events=1120 fetches.system-process.success=2 libbeat.es.call_count.PublishEvents=26 libbeat.es.publish.read_bytes=12765 libbeat.es.publish.write_bytes=690304 libbeat.es.published_and_acked_events=1258 libbeat.publisher.messages_in_worker_queues=1258 libbeat.publisher.published_events=1258
2017/11/16 05:41:27.218001 metrics.go:52: INFO Uptime: 15.258655504s
2017/11/16 05:41:27.218005 beat.go:237: INFO metricbeat stopped.
2017/11/16 05:41:27.217642 wrapper.go:201: DBG Stopped metricSetWrapper[module=system, name=diskio, host=]
2017/11/16 05:41:27.217650 wrapper.go:201: DBG Stopped metricSetWrapper[module=system, name=filesystem, host=]
2017/11/16 05:41:27.217666 wrapper.go:201: DBG Stopped metricSetWrapper[module=system, name=core, host=]
2017/11/16 05:41:27.217631 wrapper.go:201: DBG Stopped metricSetWrapper[module=system, name=load, host=]
2017/11/16 05:41:27.217672 wrapper.go:201: DBG Stopped metricSetWrapper[module=system, name=cpu, host=]
2017/11/16 05:41:27.217675 wrapper.go:201: DBG Stopped metricSetWrapper[module=postgresql, name=activity, host=172.21.197.195:5444]
2017/11/16 05:41:27.217982 metrics.go:51: INFO Total non-zero values: fetches.postgresql-activity.events=2 fetches.postgresql-activity.failures=2 fetches.postgresql-bgwriter.events=4 fetches.postgresql-bgwriter.failures=4 fetches.postgresql-database.events=2 fetches.postgresql-database.failures=2 fetches.system-core.events=12 fetches.system-core.success=2 fetches.system-cpu.events=2 fetches.system-cpu.success=2 fetches.system-diskio.events=16 fetches.system-diskio.success=2 fetches.system-filesystem.events=84 fetches.system-filesystem.success=2 fetches.system-fsstat.events=2 fetches.system-fsstat.success=2 fetches.system-load.events=2 fetches.system-load.success=2 fetches.system-memory.events=2 fetches.system-memory.success=2 fetches.system-network.events=10 fetches.system-network.success=2 fetches.system-process.events=1120 fetches.system-process.success=2 libbeat.es.call_count.PublishEvents=26 libbeat.es.publish.read_bytes=12765 libbeat.es.publish.write_bytes=690304 libbeat.es.published_and_acked_events=1258 libbeat.publisher.messages_in_worker_queues=1258 libbeat.publisher.published_events=1258
2017/11/16 05:41:27.218001 metrics.go:52: INFO Uptime: 15.258655504s
2017/11/16 05:41:27.218005 beat.go:237: INFO metricbeat stopped.

#4

pfb the metricbeat.yml


#==========================  Modules configuration ============================
metricbeat.modules:
  -
    module: system
    metricsets:
      - cpu
      - load
      - core
      - diskio
      - filesystem
      - fsstat
      - memory
      - network
      - process
    period: 10s
    processes:
      - .*
  -
    module: postgresql
    metricsets:
      - database
      - bgwriter
      - bgwriter
      - activity
    hosts:
      - "postgres://172.21.197.195:5444/SANTEDB"
    period: 10s

output.elasticsearch:
  hosts:
    - "172.21.197.152:9200"
#  password: changeme
#  username: elastic

logging.level: debug
#5

Any chance you could post a little bit a longer log and from the beginning when metricbeat starts up and then wait for about 60s? For a test run could you disable the system module?

For the log best paste it into a gist and link it here.

#6

Hi Ruflin

this is the error message i am getting, post commenting out the system module and trying only for the pgsql.

"error": "QueryStats: pq: unexpected error: "setting PGLOCALEDIR not supported"",
** "metricset":**

m2
m2.png1360×768 85.4 KB
m1
m1.png1360×768 117 KB
m2
m2.png1360×768 85.4 KB

the PGLOCALEDIR dir is already set, let me know if metricbeat expects any specific locale. the below locale are present in my system. thanks in advance
zh_TW
pt_BR
ta
ja
nb
zh_CN
it
de
id
es
fr
ro
cs
ru
sv
ko
tr
pl

#7

From the go client library we use: https://godoc.org/github.com/lib/pq

Most environment variables as specified at http://www.postgresql.org/docs/current/static/libpq-envars.html supported by libpq are also supported by pq. If any of the environment variables not supported by pq are set, pq will panic during connection establishment. Environment variables have a lower precedence than explicitly provided connection parameters.

Can you try unset the env variable?

In the future please paste text instead of screenshots.

#8

HI Ruflin,
post unsetting the PGLOCALEDIR env variable i am able to proceed with metricbeat. thanks fot the guidance.

#9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.