# Help with aggregating events

**URL:** <https://discuss.elastic.co/t/help-with-aggregating-events/116926>\
**Category:** Logstash\
**Created:** [January 24, 2018, 8:47pm UTC](https://discuss.elastic.co/t/help-with-aggregating-events/116926 "2018-01-24T20:47:46Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![TehMuffinMoo](https://avatars.discourse-cdn.com/v4/letter/t/f04885/32.png) [@TehMuffinMoo](https://discuss.elastic.co/u/TehMuffinMoo)\
**Post date:** [January 24, 2018, 8:47pm UTC](https://discuss.elastic.co/t/help-with-aggregating-events/116926/1 "2018-01-24T20:47:46Z")

</div>

Hey Guys,

My first time posting here, but I’m hoping somebody may be able to assist in pointing me in the right direction.

I’ve got OpenVPN logs going into my log server and there’s a number of events which get logged periodically. They are almost identical except for the last values which I currently assign to individual fields using the kv plugin, but this means I get 9 events each exactly the same except for one field.

An example of the batch of events is as follows;

```auto
Jan 17 13:49:53 openvpn[51830]: username/1.1.1.1:61595 peer info: IV_VER=2.4.4
Jan 17 13:49:53 openvpn[51830]: username/1.1.1.1:61595 peer info: IV_PLAT=win
Jan 17 13:49:53 openvpn[51830]: username/1.1.1.1:61595 peer info: IV_PROTO=2
Jan 17 13:49:53 openvpn[51830]: username/1.1.1.1:61595 peer info: IV_LZ4=1
Jan 17 13:49:53 openvpn[51830]: username/1.1.1.1:61595 peer info: IV_LZ4v2=1
Jan 17 13:49:53 openvpn[51830]: username/1.1.1.1:61595 peer info: IV_LZO=1
Jan 17 13:49:53 openvpn[51830]: username/1.1.1.1:61595 peer info: IV_COMP_STUB=1
Jan 17 13:49:53 openvpn[51830]: username/1.1.1.1:61595 peer info: IV_COMP_STUBv2=1
Jan 17 13:49:53 openvpn[51830]: username/1.1.1.1:61595 peer info: IV_TCPNL=1
Jan 17 13:49:53 openvpn[51830]: username/1.1.1.1:61595 peer info: IV_GUI_VER=OpenVPN_GUI_11

```

A snippet of my Grok is;

```auto
   if [program] == "openvpn" {
     grok {  
     match => ["rest", " (%{USERNAME:openvpn_user}/)?%{IP:openvpn_src_ip}:%{INT:openvpn_scr_port} (\[%{USERNAME:openvpn_user}\])?(peer info: IV\_%{NOTSPACE:openvpn_opt})?%{GREEDYDATA:openvpn_message}" ]
     }
      # OpenVPN Options
      if [openvpn_opt] != '' {
        kv {
         source => 'openvpn_opt'
         prefix => 'openvpn_opt_'
        }
      }

```

I’m hoping I can aggregate these into a single event. Any ideas how I can achieve this? 🙂

---

<div class="post-metadata">

**Author:** ![seanziee](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/seanziee/32/45151_2.png) [@seanziee](https://discuss.elastic.co/u/seanziee)\
**Post date:** [January 25, 2018, 8:55am UTC](https://discuss.elastic.co/t/help-with-aggregating-events/116926/2 "2018-01-25T08:55:45Z")

</div>

Probably your best bet is the [aggregate filter plugin](https://www.elastic.co/guide/en/logstash/current/plugins-filters-aggregate.html)!

---

<div class="post-metadata">

**Author:** ![TehMuffinMoo](https://avatars.discourse-cdn.com/v4/letter/t/f04885/32.png) [@TehMuffinMoo](https://discuss.elastic.co/u/TehMuffinMoo)\
**Post date:** [January 25, 2018, 5:04pm UTC](https://discuss.elastic.co/t/help-with-aggregating-events/116926/3 "2018-01-25T17:04:41Z")

</div>

Thanks @seanziee,

I’ll have a look this evening and see if I can work it out! I’d probably need to look at timed end events as they all come through in batches at the same time, so a short timeout should be OK. Never used this plugin so hopefully it’s not too difficult!

---

<div class="post-metadata">

**Author:** ![seanziee](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/seanziee/32/45151_2.png) [@seanziee](https://discuss.elastic.co/u/seanziee)\
**Post date:** [January 26, 2018, 3:35am UTC](https://discuss.elastic.co/t/help-with-aggregating-events/116926/4 "2018-01-26T03:35:10Z")

</div>

There's pretty good documentation on it with 4 distinct examples.

If you have a specific log that always comes last and thus points to an "END" of that string of logs, then example 1 or 2 would work for you.

But looks like most likely example 4 would work for you.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 23, 2018, 3:35am UTC](https://discuss.elastic.co/t/help-with-aggregating-events/116926/5 "2018-02-23T03:35:27Z")

</div>

Closed on February 23, 2018, 3:35am UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.