# Possible goroutine leak in sql module when database is not available

**URL:** <https://discuss.elastic.co/t/possible-goroutine-leak-in-sql-module-when-database-is-not-available/273850>\
**Category:** Beats\
**Tags:** metricbeat\
**Created:** [May 24, 2021, 4:41pm UTC](https://discuss.elastic.co/t/possible-goroutine-leak-in-sql-module-when-database-is-not-available/273850 "2021-05-24T16:41:55Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![simioa](https://avatars.discourse-cdn.com/v4/letter/s/73ab20/32.png) [@simioa](https://discuss.elastic.co/u/simioa)\
**Post date:** [May 24, 2021, 4:41pm UTC](https://discuss.elastic.co/t/possible-goroutine-leak-in-sql-module-when-database-is-not-available/273850/1 "2021-05-24T16:41:55Z")

</div>

Used metricbeat Version: 7.12.1

Hello,

we're using the metricbeat sql module to gather some metrics. During Downtime of our Database, I observed slowly increasing memory consumption from metricbeat.

I tried to reproduce this with the following metricbeat config with a non-existent Database:  
(example config from the Metricbeat reference)

```
metricbeat.modules:
- module: sql
  metricsets:
    - query
  period: 10s
  hosts: ["root:root@tcp(localhost:3306)/ps"]

  driver: "mysql"
  sql_query: "SHOW GLOBAL STATUS LIKE 'Innodb_system%'"
  sql_response_format: variables

```

While metricbeat was running, I could see a steady increase in memory consumption from metricbeat.

I restarted metricbeat with the httpprof parameter and could observe that many

> database/sql.(\*DB).connectionOpener

goroutines are being created.

```
goroutine profile: total 10395
10305 @ 0x1f2cd25 0x1f3ce8f 0x39018d5 0x1f63e41
#	0x39018d4	database/sql.(*DB).connectionOpener+0xf4	/usr/local/go/src/database/sql/sql.go:1126 

```

With debug=2

```
goroutine 2004 [select, 14 minutes]:
database/sql.(*DB).connectionOpener(0xc000df6dd0, 0x5fa9520, 0xc00096b940)
	/usr/local/go/src/database/sql/sql.go:1126 +0xf5
created by database/sql.OpenDB
	/usr/local/go/src/database/sql/sql.go:740 +0x12a

```

After some debugging I think I figured out what happens.

In [https://github.com/elastic/beats/blob/v7.12.1/metricbeat/helper/sql/sql.go#L54](https://github.com/elastic/beats/blob/v7.12.1/metricbeat/helper/sql/sql.go#L54)  
there is a check to test if the connection to the database works and if not, returns an error - but in case there is an error, the Database is never closed and leaking the goroutine.

To test my assumption, I replaced the code

```
	if err != nil {
		return nil, errors.Wrap(err, "testing connection")
	}

```

with

```
if err != nil {
	if dbx != nil {
		dbx.Close()
	}
	return nil, errors.Wrap(err, "testing connection")
}

```

After replacing the code, I could not observe further memory growth or goroutine creations.

Can you please check if my assumption is correct or if this is even a bug?

Thanks and regards,

Janni

---

<div class="post-metadata">

**Author:** ![mtojek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mtojek/32/63863_2.png) [@mtojek](https://discuss.elastic.co/u/mtojek)\
**Post date:** [May 25, 2021, 8:36am UTC](https://discuss.elastic.co/t/possible-goroutine-leak-in-sql-module-when-database-is-not-available/273850/2 "2021-05-25T08:36:32Z")

</div>

Thanks for the investigation! Would you mind opening an issue for Beats or even prepare a pull request?

---

<div class="post-metadata">

**Author:** ![simioa](https://avatars.discourse-cdn.com/v4/letter/s/73ab20/32.png) [@simioa](https://discuss.elastic.co/u/simioa)\
**Post date:** [May 26, 2021, 9:22am UTC](https://discuss.elastic.co/t/possible-goroutine-leak-in-sql-module-when-database-is-not-available/273850/3 "2021-05-26T09:22:05Z")

</div>

I created an issue: [Possible goroutine leak in metricbeat sql module when database is not available · Issue #25840 · elastic/beats (github.com)](https://github.com/elastic/beats/issues/25840)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 23, 2021, 11:22am UTC](https://discuss.elastic.co/t/possible-goroutine-leak-in-sql-module-when-database-is-not-available/273850/4 "2021-06-23T11:22:48Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.