Unable to connect to postgres module from metricbeat

Dinesh_Senthil_Kumar · August 16, 2017, 8:02am

hi ALL,
i ma getting the bewlo error while connecting to postgres module from metric beat. need your help or pointers to resolve the same. currently i am using metric beat 5.4.x and postgres 9.5.05.

2017/08/16 08:02:16.287078 beat.go:221: INFO metricbeat start running.
2017/08/16 08:02:16.291421 single.go:140: ERR Connecting error publishing events (retrying): Get postgres://localhost:5450: unsupported protocol scheme "postgres"
2017/08/16 08:02:17.291688 single.go:140: ERR Connecting error publishing events (retrying): Get postgres://localhost:5450: unsupported protocol scheme "postgres"
2017/08/16 08:02:19.291948 single.go:140: ERR Connecting error publishing events (retrying): Get postgres://localhost:5450: unsupported protocol scheme "postgres"

the connection string that i am using

  module: postgresql
  metricsets:
        - database
        - bgwriter
        - activity
  period: 10s
  hosts: ["postgres://localhost:5450"]

andrewkroh · August 16, 2017, 1:08pm

It looks to me like the documentation is wrong. The scheme on the URL should be postgresql:// according to https://www.postgresql.org/docs/current/static/libpq-connect.html#LIBPQ-CONNSTRING. Could you please confirm that this fixes the issue and if so open an issue on Github so that we can update the docs and config files.

Dinesh_Senthil_Kumar · August 16, 2017, 1:30pm

HI Andrew,

Tried as suggested but it is not working, i ma still getting the below error.

2017/08/16 13:30:14.381830 beat.go:221: INFO metricbeat start running.
2017/08/16 13:30:14.385282 single.go:140: ERR Connecting error publishing events (retrying): Get postgresql://localhost:5450: unsupported protocol scheme "postgresql"
2017/08/16 13:30:15.385807 single.go:140: ERR Connecting error publishing events (retrying): Get postgresql://localhost:5450: unsupported protocol scheme "postgresql"
2017/08/16 13:30:17.386128 single.go:140: ERR Connecting error publishing events (retrying): Get postgresql://localhost:5450: unsupported protocol scheme "postgresql"
2017/08/16 13:30:21.386380 single.go:140: ERR Connecting error publishing events (retrying): Get postgresql://localhost:5450: unsupported protocol scheme "postgresql"

url used :hosts: ["postgresql://localhost:5450"]

andrewkroh · August 16, 2017, 1:46pm

Please share your full metricbeat configuration.

Dinesh_Senthil_Kumar · August 17, 2017, 6:43am

hi pfb the metricbeat config

###################### Metricbeat Configuration Example #######################

# This file is an example configuration file highlighting only the most common
# options. The metricbeat.full.yml file from the same directory contains all the
# supported options with more comments. You can use it as a reference.
#
# You can find the full configuration reference here:
# https://www.elastic.co/guide/en/beats/metricbeat/index.html

#==========================  Modules configuration ============================
metricbeat.modules:

#------------------------------- System Module -------------------------------
- module: system
  metricsets:
    # CPU stats
    - cpu

    # System Load stats
    - load

    # Per CPU core stats
    - core

    # IO stats
    - diskio

    # Per filesystem stats
    - filesystem

    # File system summary stats
    - fsstat

    # Memory stats
    - memory

    # Network stats
    - network

    # Per process stats
    - process

    # Sockets (linux only)
    #- socket
  enabled: true
  period: 10s
  processes: ['.*']



#================================ General =====================================

# The name of the shipper that publishes the network data. It can be used to group
# all the transactions sent by a single shipper in the web interface.
#name:

# The tags of the shipper are included in their own field with each
# transaction published.
#tags: ["service-X", "web-tier"]

# Optional fields that you can specify to add additional information to the
# output.
#fields:
#  env: staging

#================================ Outputs =====================================

# Configure what outputs to use when sending the data collected by the beat.
# Multiple outputs may be used.

#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["172.21.197.156:9200"]

  # Optional protocol and basic auth credentials.
  #protocol: "https"
  username: "elastic"
  password: "changeme"
#----------------------------- PostgreSQL Module -----------------------------
  module: postgresql
  metricsets:
#      # Stats about every PostgreSQL database
        - database
#
#              # Stats about the background writer process's activity
        - bgwriter
        - bgwriter
#
#                      # Stats about every PostgreSQL process
        - activity
#
  enabled: true
  period: 10s
                                # The host must be passed as PostgreSQL URL. Example:
 # postgres: //localhost:5450?sslmode=disable
#                                    # The available parameters are documented here:
#                                      # https://godoc.org/github.com/lib/pq#hdr-Connection_String_Parameters
#  hosts: ["postgres://localhost:5450?sslmode=disable"]
#  hosts:  ["postgres://localhost:5450"]
#  https://godoc.org/github.com/lib/pq#hdr-Connection_String_Parameters
  hosts:  ["postgres://localhost:5450/LMS114DB"]

#                                          # Username to use when connecting to PostgreSQL. Empty by default.
#  username: enterprisedb
#
#                                              # Password to use when connecting to PostgreSQL. Empty by default.
#  password: infy123
#

#----------------------------- Logstash output --------------------------------
#output.logstash:
  # The Logstash hosts
  #hosts: ["localhost:5044"]

  # Optional SSL. By default is off.
  # List of root certificates for HTTPS server verifications
  #ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]

  # Certificate for SSL client authentication
  #ssl.certificate: "/etc/pki/client/cert.pem"

  # Client Certificate Key
  #ssl.key: "/etc/pki/client/cert.key"

#================================ Logging =====================================

# Sets log level. The default log level is info.
# Available log levels are: critical, error, warning, info, debug
logging.level: debug

# At debug level, you can selectively enable logging only for some components.
# To enable all selectors use ["*"]. Examples of other selectors are "beat",
# "publish", "service".
#logging.selectors: ["*"]

andrewkroh · August 17, 2017, 12:43pm

You have the postgresql module config in the wrong place. The metricbeat.modules option is a list and all the items in the list need to be listed together. Try this config:

--- 
metricbeat.modules: 
  - 
    module: system
    metricsets: 
      - cpu
      - load
      - core
      - diskio
      - filesystem
      - fsstat
      - memory
      - network
      - process
    period: 10s
    processes: 
      - .*
  - 
    module: postgresql
    metricsets: 
      - database
      - bgwriter
      - bgwriter
      - activity
    hosts: 
      - "postgres://localhost:5450/LMS114DB"
    period: 10s

output.elasticsearch: 
  hosts: 
    - "172.21.197.156:9200"
  password: changeme
  username: elastic

logging.level: debug

Dinesh_Senthil_Kumar · August 17, 2017, 1:48pm

thanks andrew this is working,appreciate your quick help

system · September 14, 2017, 1:48pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.