Limit database connections

[Daniel_DNP]

Hello,

I'm writing a metricbeat module for Oracle Database (module: oracledb) monitoring.

For each metricset (e.g. status, fra, ...) one database connection is opened. I try to reduce the number of connections because I want to avoid 30 or more database connections. So I tried to share the connection among the metricsets. I created a map on package level in oracledb
This seems not to work. I have the impression that metricbeat calls all metricsets in different "threads". This means that each metricset has a different instance of oracledb.

oracledb.yml:
oracledb.modules:
- module: oracledb
metricsets: ["fra", "tablespace" ]
enabled: true
period: 10s
hosts: ["scott/tiger@localhost:1521/orcl"]

Is this correct?
How can I limit the number of connections to the database?

Thanks a lot!

Daniel

[steffens (Steffen Siering)]

Metricbeat uses a go-routine per metricset. Some connection pooling with timeouts (to close idle connections) on package level should help though.

e.g. pool in oracledb package could have type:

var pool connectionPool

type connectionPool struct {
  mutex sync.Mutex
  maxConnections int
  pools map[connectInfo]DB
}

// if all fields in connectInfo are hashable, the struct can be used as key
type connectInfo struct {
  Address string
  Username string
  Password string // maybe hash the password string here?
}

type DB struct {
  mutex sync.Mutex
  connections []connection
}

type connection struct {
  ...
}

func (c *connection) Close() error {
  // TODO: return to pool + start actual close timeout
}

If you just want to restrict total connection count another solution can be a semaphore-like lock (e.g. using sync.Cond).

var (
  mutex sync.Mutex
  cond = sync.NewCond(&mutex)
  maxConnections int
  connectionCount int
)

func Open(connectionDetails) (*Connection, error) {
  reserve()

  // open connection, call release() on error, so other go-routines can establish a connection
}

func (c *Connection) Close() error {
  err := c.db.Close()
  release()
  return err
}

func reserve() {
  mutex.Lock()
  defer mutex.Unlock()
  for !(connectionCount < maxConnections) {
    cond.Wait()
  }
  connectionCount++
}

func release() {
  mutex.Lock()
  connectionCount--
  mutex.Unlock()

  // wake one waiting go-routine
  cond.Signal()
}

Unfortunately sync.Cond does not support timeouts. In case you want to add timeouts, you'd have to use channels one way or the other.

Given one normally monitors services on localhost, and you have configured 2 metricsets only, how come you will end up with 30 connections?