Is there a goroutine leak in libbeat?

[erenming (Eren)]

I'm doing a pressure test for our beats like filebeat use libbeat package，But I found github.com/elastic/beats/libbeat/reader/readfile.(*TimeoutReader).Next.func1 is continuous growing。

The presure test condition is described as follows：

1. i generate 100 file concurrently
2. and generate log line without sleep for about five seconds
3. limit the container with 0.2 cpus, 512Mi.

this is memory and cpu utilization:

and this is the goroutine compares, and the duration is about 30minutes.

and the version of libbeat is v7.6.2

anyone could help me? thanks

[ChrsMark (Chris Mark)]

Hi!

It seems that the memory is constantly going up, which could be a bad indicator indeed.
From what I understand from your test there are logs coming only for the first 5 seconds and then nothing? Or you are constantly adding batches of 100 files with a specific rate?

cc: @kvch

[erenming (Eren)]

Thanks for your reply!
This is my pressure test procedure.
First, i create 100 files, and then add about 5000 lines to each file in 5 seconds。

[ChrsMark (Chris Mark)]

Hi @erenming!

I suggest you opening a Github issue to report this observation since it sounds valid enough. Could you please open a Github issue in order to keep track of this please? Provide all the information you posted here as well as possible steps to reproduce.

Thanks!