# Metricbeat 5.5 sending events to Logstash in batches

**URL:** <https://discuss.elastic.co/t/metricbeat-5-5-sending-events-to-logstash-in-batches/98269>\
**Category:** Beats\
**Tags:** metricbeat\
**Created:** [August 24, 2017, 5:05pm UTC](https://discuss.elastic.co/t/metricbeat-5-5-sending-events-to-logstash-in-batches/98269 "2017-08-24T17:05:17Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![keithn](https://avatars.discourse-cdn.com/v4/letter/k/8491ac/32.png) [@keithn](https://discuss.elastic.co/u/keithn)\
**Post date:** [August 24, 2017, 5:05pm UTC](https://discuss.elastic.co/t/metricbeat-5-5-sending-events-to-logstash-in-batches/98269/1 "2017-08-24T17:05:17Z")

</div>

Is it possible for Metricbeat to send multiple events to Logstash in batches?

For example, collect system CPU and Memory metrics every second, but ship the metrics to Logstash in batches (e.g. 60 events) every minute? Thanks.

Sample config:

```
metricbeat.modules:
- module: system
  metricsets:
    - cpu
    - memory
  enabled: true
  period: 1s
  processes: ['.*']

output.logstash:
  hosts: ["localhost:5044"]
```

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [August 24, 2017, 6:21pm UTC](https://discuss.elastic.co/t/metricbeat-5-5-sending-events-to-logstash-in-batches/98269/2 "2017-08-24T18:21:02Z")

</div>

Why do you want to do this? I believe the outputs support setting the `flush_interval` (default is 1s). You could set this to 60s then internally queued events would be sent every 60s or when the queue reaches it max size (configured with `bulk_max_size` and defaults to 2048).

---

<div class="post-metadata">

**Author:** ![keithn](https://avatars.discourse-cdn.com/v4/letter/k/8491ac/32.png) [@keithn](https://discuss.elastic.co/u/keithn)\
**Post date:** [August 24, 2017, 6:52pm UTC](https://discuss.elastic.co/t/metricbeat-5-5-sending-events-to-logstash-in-batches/98269/3 "2017-08-24T18:52:14Z")

</div>

Thanks Andrew. This is actually what I was looking for. I didn't know the "flush\_interval" setting exist for the logstash output.

---

<div class="post-metadata">

**Author:** ![keithn](https://avatars.discourse-cdn.com/v4/letter/k/8491ac/32.png) [@keithn](https://discuss.elastic.co/u/keithn)\
**Post date:** [August 25, 2017, 2:35pm UTC](https://discuss.elastic.co/t/metricbeat-5-5-sending-events-to-logstash-in-batches/98269/4 "2017-08-25T14:35:59Z")

</div>

@andrewkroh, will Metricbeat send the internally queued events to Logstash as a single request? Thanks.

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [August 25, 2017, 3:05pm UTC](https://discuss.elastic.co/t/metricbeat-5-5-sending-events-to-logstash-in-batches/98269/5 "2017-08-25T15:05:12Z")

</div>

It should send the events in a batch. The batch size may start small and grow once it determines the Logstash is able to handle the larger batch sizes (this behavior [was disabled in 6.0](https://github.com/elastic/beats/pull/4972)).

---

<div class="post-metadata">

**Author:** ![keithn](https://avatars.discourse-cdn.com/v4/letter/k/8491ac/32.png) [@keithn](https://discuss.elastic.co/u/keithn)\
**Post date:** [August 30, 2017, 8:51pm UTC](https://discuss.elastic.co/t/metricbeat-5-5-sending-events-to-logstash-in-batches/98269/6 "2017-08-30T20:51:58Z")

</div>

Does Metricbeat average out the metrics for CPU, Memory, Network, etc. over the period (e.g. 10s) set in the config file? Or does it collect raw metrics at the time? Thanks.

For example, collecting every 10s:

```
- module: system
  metricsets:
    - cpu
    - memory
    - network
  enabled: true
  period: 10s
  processes: ['.*']
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 27, 2017, 8:51pm UTC](https://discuss.elastic.co/t/metricbeat-5-5-sending-events-to-logstash-in-batches/98269/7 "2017-09-27T20:51:58Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.