# Aggregate function help

**URL:** <https://discuss.elastic.co/t/aggregate-function-help/343432>\
**Category:** Logstash\
**Created:** [September 20, 2023, 8:30am UTC](https://discuss.elastic.co/t/aggregate-function-help/343432 "2023-09-20T08:30:42Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ameeruddin\_Mohammed](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ameeruddin_mohammed/32/41289_2.png) [@Ameeruddin\_Mohammed](https://discuss.elastic.co/u/Ameeruddin_Mohammed)\
**Post date:** [September 20, 2023, 8:30am UTC](https://discuss.elastic.co/t/aggregate-function-help/343432/1 "2023-09-20T08:30:43Z")

</div>

hi,  
i have logs in this format and i want to start the aggregation when start comes in the line and end the aggregation when end occurs. the aggregation is based on "username".  
i was able to use aggregate function but it does not split the documents.  
username;command;start/end  
user1;command1;  
user1;command2;start  
user2;command3;start  
user3;command4;  
user1;command5;  
user2;command6;  
user3;command7;start  
user2;command8;end  
user1;command9;  
user2;command10;end  
user3;command11;end  
user2;command12;  
user1;command13;end  
user2;command14;  
user1;command15;  
user3;command16;

needed  
doc1  
user1;command1;

doc2  
user1;command2;start  
user1;command5;  
user1;command9;  
user1;command13;end

doc3  
user1;command15;

doc4  
user2;command3;start  
user2;command6;  
user2;command8;end

doc5  
user2;command12;  
user2;command14;

doc6  
user3;command4;

doc7  
user3;command7;start  
user3;command11;

doc8  
user3;command16;

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [September 20, 2023, 7:19pm UTC](https://discuss.elastic.co/t/aggregate-function-help/343432/2 "2023-09-20T19:19:35Z")

</div>

You want command12 and command14 aggregated even though there is no start/end, but you do not want command1 aggregated? So until a start appears for a user you do not want aggregation, but once a start has been seen you want everything aggregated?

---

<div class="post-metadata">

**Author:** ![Ameeruddin\_Mohammed](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ameeruddin_mohammed/32/41289_2.png) [@Ameeruddin\_Mohammed](https://discuss.elastic.co/u/Ameeruddin_Mohammed)\
**Post date:** [September 20, 2023, 9:54pm UTC](https://discuss.elastic.co/t/aggregate-function-help/343432/3 "2023-09-20T21:54:15Z")

</div>

Hi,  
Sorry my bad, those two should be separate documents.  
Yes, all documents should be individually captured. Once start starts aggregation should happen and aggregation should end once end is found.  
Post end till new start per user no aggregation.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [September 20, 2023, 11:55pm UTC](https://discuss.elastic.co/t/aggregate-function-help/343432/4 "2023-09-20T23:55:05Z")

</div>

I tried doing this with three aggregate filters (similar to [example 1](https://www.elastic.co/guide/en/logstash/current/plugins-filters-aggregate.html#plugins-filters-aggregate-example1) in the documentation) and could not get it to work. I then decided to do it in ruby.

```
     csv {
        columns => ["[@metadata][user]", "[@metadata][command]", "[@metadata][startEnd]" ]
        separator => ";"
    }

    ruby {
        code => '
            @users ||= {}

            meta = event.get("@metadata")
            user = meta["user"]
            @users[user] ||= {}
            msg = event.get("message")

            if meta["startEnd"] == "start/end"
                event.cancel
            elsif meta["startEnd"] == "start"
                @users[user] = { "started" => true, "commands" => [msg] }
                event.cancel
            elsif meta["startEnd"] == "end"
                @users[user]["commands"] << msg
                event.set("commands", @users[user]["commands"])
                @users[user].delete("started")
            else
                if @users[user].has_key?("started")
                    @users[user]["commands"] << msg
                    event.cancel
                else
                    event.set("commands", [msg])
                end
            end
        '
    }

```

will do the wrong thing if the start/end messages are not paired up, but it's close.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 18, 2023, 11:56pm UTC](https://discuss.elastic.co/t/aggregate-function-help/343432/5 "2023-10-18T23:56:00Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.