Regarding pipeline.buffer.type

Elastic Stack Logstash
1

Hi,
Recently we've been enabling heap based processing of the beats input as you can see here:

pipeline.workers: 2
pipeline.batch.size: 100
pipeline.buffer.type: heap

However, we still see errors for some sites which suggest otherwise:

[2025-07-04T04:31:09,981][INFO ][org.logstash.beats.BeatsHandler][default-input][beats-input] [local: <ip-address>:5044, remote: <ip-address>:50370] Handling exception: java.lang.OutOfMemoryError: Cannot reserve 67108864 bytes of direct buffer memory (allocated: 3490177720, limit: 3552575488) (caused by: java.lang.OutOfMemoryError: Cannot reserve 67108864 bytes of direct buffer memory (allocated: 3490177720, limit: 3552575488))
[2025-07-04T05:46:39,131][WARN ][org.logstash.beats.V2Batch][default-input][beats-input] Received batch of size 69052687 bytes that is too large to fit into the pre-allocated memory pool. This will cause a performance degradation. Set 'io.netty.allocator.maxOrder' JVM property to 14 to accommodate batches bigger than 67108864 bytes.

The version we're running is 8.16.3. Are we doing something wrong here? Or is it something else?

Thx
D

2

Have you tried to add line: -Dio.netty.allocator.maxOrder=14 or 15 in jvm.options & restart LS.

What is a little bit weird, you set, however it's direct buffer memory has been recorded in the log.

3

That's what I'm querying here. Why is it logging direct memory warnings when it's configured to use heap?

4

Strangely, for me, it defaults to heap unless set to direct!

ruby { code => 'java_import "java.lang.System"; puts System.getProperties()["io.netty.noPreferDirect"]' }

prints true unless I uncomment the "# pipeline.buffer.type: direct" line in logstash.yml

5

Docs >> pipeline.buffer.type Determine where to allocate memory buffers, for plugins that leverage them.Defaults to heap but can be switched to direct to instruct Logstash to prefer allocation of buffers in direct memory.
This setting exist from 8.16.0.

Dawiro, have you restarted LS after changes? Check GH thread this and this
Also, check LS log in the debug mode, should be a trace which value has been set.

6

The code suggests that it defaults to direct. It seems pipeline.buffer.type is getting set somewhere other than logstash.yml , or else I am massively confused.

Today I learned something new about how settings work. This is the code that sets the default, which became heap in v9.0.

1 Like
7

@Rios @Badger We've been running 8.16.x for a while now. We're deploying to aws via terraform so these aren't manual changes.

Also, I believe the setting was brought in defaulting to direct in 8.16.x if I recall. That was what motivated me to upgrade to that release, so that we could switch to heap mode.