# Need to aggregate data in a CSV file based on a key

**URL:** https://discuss.elastic.co/t/need-to-aggregate-data-in-a-csv-file-based-on-a-key/378642
**Category:** Logstash
**Created:** [May 28, 2025, 11:01am UTC](https://discuss.elastic.co/t/need-to-aggregate-data-in-a-csv-file-based-on-a-key/378642 "2025-05-28T11:01:09Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![venkatkumar229](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/venkatkumar229/32/104663_2.png) [@venkatkumar229](https://discuss.elastic.co/u/venkatkumar229)
#### Post date: [May 28, 2025, 11:01am UTC](https://discuss.elastic.co/t/need-to-aggregate-data-in-a-csv-file-based-on-a-key/378642/1 "2025-05-28T11:01:09Z")

</div>

Hi Everyone,

I have an use case where i need to aggregate the lines in a CSV file to get the sum of amount, quantity of an order based on the ordernumber. I have tried to use "aggregate" filter but i am end up getting one record for unique ordernumber but the values are not getting added instead i am getting last values for the ordernumber.

Sample csv data:

```auto
order_number,item_id,item_name,quantity,price
1001,A1,Item1,2,10
1001,A2,Item2,1,15
1002,B1,Item3,3,5

```

Desired Output:

```auto
{
  "order_number": "1001",
  "total_price": 25.0,
  "total_quantity": 3,
},
{
  "order_number": "1002",
  "total_price": 5.0,
  "total_quantity": 3,
}

```

What i am getting:( Getting the last latest values of price, quantity for the ordernumber instead of sum)

```auto
{
  "order_number": "1001",
  "total_price": 15.0,
  "total_quantity": 1,
},
{
  "order_number": "1002",
  "total_price": 5.0,
  "total_quantity": 3,
}

```

My aggregate filter:

```auto
filter {
  aggregate {
    task_id => "%{order_number}"
    code => "
      quan = event.get('quantity') ? event.get('quantity').to_i : 0
      price = event.get('price') ? event.get('price').to_f : 0.0

      map['total_quantity'] ||= 0
      map['total_price'] ||= 0.0

      map['total_quantity'] += quan
      map['total_price'] += quan * price

      map['order_number'] = event.get('order_number')
    "
    push_map_as_event_on_timeout => true
    timeout_task_id_field => "order_number"
    timeout => 10
    timeout_tags => ["aggregated"]
  }

  if !("aggregated" in [tags]) {
    drop {}
  }
}

```

I have used the pipeline workers as 1.

Please help me to solve this issue.

Thanks in advance.

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [May 28, 2025, 4:42pm UTC](https://discuss.elastic.co/t/need-to-aggregate-data-in-a-csv-file-based-on-a-key/378642/2 "2025-05-28T16:42:01Z")

</div>

If I use `csv { autodetect_column_names => true }` then I get 35.0 and 15.0 for [total\_price]. I see nothing wrong with the aggregate filter.

---

<div class="post-metadata">

### Author: ![venkatkumar229](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/venkatkumar229/32/104663_2.png) [@venkatkumar229](https://discuss.elastic.co/u/venkatkumar229)
#### Post date: [May 29, 2025, 6:09am UTC](https://discuss.elastic.co/t/need-to-aggregate-data-in-a-csv-file-based-on-a-key/378642/3 "2025-05-29T06:09:26Z")

</div>

Hi @Badger, Thankyou for the input. Let me try with the suggestion.
