# How to set \`bulk\_max\_size\` and \`compression\_level\`?

**URL:** <https://discuss.elastic.co/t/how-to-set-bulk-max-size-and-compression-level/341387>\
**Category:** Elastic Agent\
**Tags:** fleet, filebeat\
**Created:** [August 22, 2023, 2:52pm UTC](https://discuss.elastic.co/t/how-to-set-bulk-max-size-and-compression-level/341387 "2023-08-22T14:52:29Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Craig\_Rodrigues](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/craig_rodrigues/32/121875_2.png) [@Craig\_Rodrigues](https://discuss.elastic.co/u/Craig_Rodrigues)\
**Post date:** [August 22, 2023, 2:52pm UTC](https://discuss.elastic.co/t/how-to-set-bulk-max-size-and-compression-level/341387/1 "2023-08-22T14:52:29Z")

</div>

I have about 2000 Elastic agents (version 8.9.0) connected to a system with 3 Fleet servers (version 8.9.0).

We have about 20 different agent policies, because the various Elastic agents are sending  
slightly different logs, and for certain cases we need to specify specific pipelines to process the logs.

At this link: [Configure the Elasticsearch output | Fleet and Elastic Agent Guide [8.9] | Elastic](https://www.elastic.co/guide/en/fleet/current/elasticsearch-output.html#output-elasticsearch-performance-tuning-settings)

I see descriptions for `bulk_max_size` and `compression_level`.

How can I set those in the Elastic Agent policy in the Fleet UI?

I looked in the Fleet UI for editing an agent policy at:  
kbn:/app/fleet/policies/{policy-id}/settings  
and did not see a place where I could modify those fields

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [August 22, 2023, 4:13pm UTC](https://discuss.elastic.co/t/how-to-set-bulk-max-size-and-compression-level/341387/2 "2023-08-22T16:13:52Z")

</div>

Hi @Craig_Rodrigues

It is edited at the output not the policy

Fleet -\> Setting -\> Output

In the yaml settings

 ![Screenshot 2023-08-22 at 9.11.44 AM](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ab8a031294c3b3aa1d3ff4c40a4f4c4f2aca184.jpeg)

---

<div class="post-metadata">

**Author:** ![Craig\_Rodrigues](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/craig_rodrigues/32/121875_2.png) [@Craig\_Rodrigues](https://discuss.elastic.co/u/Craig_Rodrigues)\
**Post date:** [August 22, 2023, 4:20pm UTC](https://discuss.elastic.co/t/how-to-set-bulk-max-size-and-compression-level/341387/3 "2023-08-22T16:20:22Z")

</div>

@stephenb Thanks for the response.  
I have 2000 agents using 20 different agent policies.

In my use case, I only want certain agents to be affected by the `bulk_max_size` and `compression_level` settings.

I do not want these settings to apply to all 2000 agents.

Is that possible, or can I only specify these settings globally?

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [August 22, 2023, 4:43pm UTC](https://discuss.elastic.co/t/how-to-set-bulk-max-size-and-compression-level/341387/4 "2023-08-22T16:43:56Z")

</div>

You can create different outputs one with settings one without and apply them as needed.

You will need to check of output is per policy or agent I don't recall AFK right now.

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [August 22, 2023, 4:47pm UTC](https://discuss.elastic.co/t/how-to-set-bulk-max-size-and-compression-level/341387/5 "2023-08-22T16:47:29Z")

</div>

> [@Craig\_Rodrigues](#):
>
> I do not want these settings to apply to all 2000 agents.
> 
> Is that possible, or can I only specify these settings globally?

The output is per policy, so it will be applied on every agent under that specific policy.

---

<div class="post-metadata">

**Author:** ![Craig\_Rodrigues](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/craig_rodrigues/32/121875_2.png) [@Craig\_Rodrigues](https://discuss.elastic.co/u/Craig_Rodrigues)\
**Post date:** [August 22, 2023, 6:27pm UTC](https://discuss.elastic.co/t/how-to-set-bulk-max-size-and-compression-level/341387/6 "2023-08-22T18:27:11Z")

</div>

OK, so based on the advice @stephenb and @leandrojmp have given, I think I need to do this:

1. Go to Fleet -\> Settings  
( kbn:/app/fleet/settings )

2. Go to `Outputs` section. Right now, I only have one Output, labelled default:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/7/e/7ee1af5f4dcc31f9c7c927aafc62578142d41d11.png)

1. Click on `Add output` to add another output

2. In the new output go to Advanced YAML configuration, add the settings for `bulk_max_size` and `compression_level` there:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/7/8/7862531a9388f6bf6bd3ebcf9e474a9bb308eaad.png)

1. Go to a particular agent policy  
Fleet -\> Agent policies  
( kbn:/app/fleet/policies/{policy id} )

2. Scroll down to `Output for integrations`

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/b/6/b6231fa2b73394473c23f02421b4c2d9145f0c00.png)

1. Change this output to point to the output I created in step 3.

Some of this is documented at [Advanced YAML configuration](https://www.elastic.co/guide/en/fleet/current/fleet-settings.html#:~:text=setting%20is%20required.-,Advanced%20YAML%20configuration,-YAML%20settings%20that)

Does my understanding of the necessary steps seem right?

---

<div class="post-metadata">

**Author:** ![Craig\_Rodrigues](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/craig_rodrigues/32/121875_2.png) [@Craig\_Rodrigues](https://discuss.elastic.co/u/Craig_Rodrigues)\
**Post date:** [August 24, 2023, 10:52am UTC](https://discuss.elastic.co/t/how-to-set-bulk-max-size-and-compression-level/341387/7 "2023-08-24T10:52:53Z")

</div>

@leandrojmp @stephenb Do the steps which I have described above seem right?

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [August 24, 2023, 12:40pm UTC](https://discuss.elastic.co/t/how-to-set-bulk-max-size-and-compression-level/341387/8 "2023-08-24T12:40:30Z")

</div>

Not sure what I could add, this is exactly what it is present in the documentation.

If you want this to just some agents you will need to first create the new output with this configuration, then create a new policy and use this output.

---

<div class="post-metadata">

**Author:** ![Craig\_Rodrigues](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/craig_rodrigues/32/121875_2.png) [@Craig\_Rodrigues](https://discuss.elastic.co/u/Craig_Rodrigues)\
**Post date:** [August 31, 2023, 6:34pm UTC](https://discuss.elastic.co/t/how-to-set-bulk-max-size-and-compression-level/341387/9 "2023-08-31T18:34:19Z")

</div>

Thanks for your help @leandrojmp and @stephenb

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 28, 2023, 6:35pm UTC](https://discuss.elastic.co/t/how-to-set-bulk-max-size-and-compression-level/341387/10 "2023-09-28T18:35:06Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
