# Regarding pipeline.buffer.type **URL:** \ **Category:** Logstash\ **Created:** [July 4, 2025, 7:51am UTC](https://discuss.elastic.co/t/regarding-pipeline-buffer-type/379798 "2025-07-04T07:51:36Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![dawiro](https://avatars.discourse-cdn.com/v4/letter/d/71e660/32.png) [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Post date:** [July 4, 2025, 7:51am UTC](https://discuss.elastic.co/t/regarding-pipeline-buffer-type/379798/1 "2025-07-04T07:51:36Z")
Hi, Recently we've been enabling heap based processing of the beats input as you can see here: ```auto pipeline.workers: 2 pipeline.batch.size: 100 pipeline.buffer.type: heap ``` However, we still see errors for some sites which suggest otherwise: ```auto [2025-07-04T04:31:09,981][INFO][org.logstash.beats.BeatsHandler][default-input][beats-input] [local: :5044, remote: :50370] Handling exception: java.lang.OutOfMemoryError: Cannot reserve 67108864 bytes of direct buffer memory (allocated: 3490177720, limit: 3552575488) (caused by: java.lang.OutOfMemoryError: Cannot reserve 67108864 bytes of direct buffer memory (allocated: 3490177720, limit: 3552575488)) [2025-07-04T05:46:39,131][WARN][org.logstash.beats.V2Batch][default-input][beats-input] Received batch of size 69052687 bytes that is too large to fit into the pre-allocated memory pool. This will cause a performance degradation. Set 'io.netty.allocator.maxOrder' JVM property to 14 to accommodate batches bigger than 67108864 bytes. ``` The version we're running is 8.16.3. Are we doing something wrong here? Or is it something else? Thx D ---
**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\ **Post date:** [July 4, 2025, 9:15am UTC](https://discuss.elastic.co/t/regarding-pipeline-buffer-type/379798/2 "2025-07-04T09:15:44Z")
Have you tried to add line: `-Dio.netty.allocator.maxOrder=14` or 15 in `jvm.options` & restart LS. What is a little bit weird, you set, however it's direct buffer memory has been recorded in the log. ---
**Author:** ![dawiro](https://avatars.discourse-cdn.com/v4/letter/d/71e660/32.png) [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Post date:** [July 4, 2025, 1:39pm UTC](https://discuss.elastic.co/t/regarding-pipeline-buffer-type/379798/3 "2025-07-04T13:39:38Z")
That's what I'm querying here. Why is it logging direct memory warnings when it's configured to use heap? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [July 4, 2025, 2:59pm UTC](https://discuss.elastic.co/t/regarding-pipeline-buffer-type/379798/4 "2025-07-04T14:59:33Z")
Strangely, for me, it defaults to heap unless set to direct! ``` ruby { code => 'java_import "java.lang.System"; puts System.getProperties()["io.netty.noPreferDirect"]' } ``` prints true unless I uncomment the "# pipeline.buffer.type: direct" line in logstash.yml ---
**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\ **Post date:** [July 4, 2025, 4:00pm UTC](https://discuss.elastic.co/t/regarding-pipeline-buffer-type/379798/5 "2025-07-04T16:00:18Z")
> [@Badger](#): > > prints true unless I uncomment the "# pipeline.buffer.type: direct" line in logstash.yml Docs \>\> `pipeline.buffer.type` Determine where to allocate memory buffers, for plugins that leverage them.Defaults to `heap` but can be switched to `direct` to instruct Logstash to prefer allocation of buffers in direct memory. This setting exist from [8.16.0](https://www.elastic.co/guide/en/logstash/8.17/logstash-8-16-0.html). Dawiro, have you restarted LS after changes? Check GH thread [this](https://github.com/elastic/logstash/pull/16054) and [this](https://github.com/elastic/logstash/pull/16498) Also, check LS log in the debug mode, should be a trace which value has been set. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [July 4, 2025, 4:22pm UTC](https://discuss.elastic.co/t/regarding-pipeline-buffer-type/379798/6 "2025-07-04T16:22:37Z")
> [@Rios](#): > > Defaults to `heap` but can be switched to `direct` The [code](https://github.com/andsel/logstash/blob/b0c73c91a31479a6ddb853fa03175afd1b5f9dcb/logstash-core/lib/logstash/runner.rb#L610) suggests that it defaults to direct. It seems pipeline.buffer.type is getting set somewhere other than logstash.yml ~~, or else I am massively confused~~. Today I learned something new about how settings work. [This](https://github.com/elastic/logstash/blob/e9a3b7edbab822933eea71c062c223a4a81343a1/logstash-core/lib/logstash/environment.rb#L106) is the code that sets the default, which became heap in v9.0. ---
**Author:** ![dawiro](https://avatars.discourse-cdn.com/v4/letter/d/71e660/32.png) [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Post date:** [July 9, 2025, 7:55am UTC](https://discuss.elastic.co/t/regarding-pipeline-buffer-type/379798/7 "2025-07-09T07:55:44Z")
@Rios @Badger We've been running 8.16.x for a while now. We're deploying to aws via terraform so these aren't manual changes. Also, I believe the setting was brought in defaulting to `direct` in 8.16.x if I recall. That was what motivated me to upgrade to that release, so that we could switch to `heap` mode. ---
**Author:** ![dawiro](https://avatars.discourse-cdn.com/v4/letter/d/71e660/32.png) [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Post date:** [July 18, 2025, 8:49am UTC](https://discuss.elastic.co/t/regarding-pipeline-buffer-type/379798/8 "2025-07-18T08:49:12Z")
Is what I'm seeing a bug that needs fixing?