# Aggregate filter not working correctly iwth logstash
**URL:** https://discuss.elastic.co/t/aggregate-filter-not-working-correctly-iwth-logstash/185088
**Category:** Logstash
**Created:** [June 11, 2019, 4:40am UTC](https://discuss.elastic.co/t/aggregate-filter-not-working-correctly-iwth-logstash/185088 "2019-06-11T04:40:56Z")
**Posts on this page:** 3
**Page:** 1
### Author:  [@msk\_76](https://discuss.elastic.co/u/msk_76)
#### Post date: [June 11, 2019, 4:40am UTC](https://discuss.elastic.co/t/aggregate-filter-not-working-correctly-iwth-logstash/185088/1 "2019-06-11T04:40:56Z")
Here is my input log file names as : cadence-cdslmd-dlhl0939\_dlhl0940\_dlhl0941-5.11.2019-event.log
15:56:24 (cdslmd) OUT: "PKC0603" mnarasim@dlhsx00012
15:56:32 (cdslmd) OUT: "Analog\_Design\_Environment\_XL" gautamd@dlhsx00005
15:56:33 (lmgrd) TIMESTAMP 5/13/2019
15:56:35 (cdslmd) OUT: "OASIS\_Simulation\_Interface" mnarasim@dlhsx00012
15:56:37 (cdslmd) IN: "111" guptasri@dlhsx00010
15:56:42 (cdslmd) OUT: "PKC0603" mnarasim@dlhsx00012
My logstash filter
input {
beats {
port =\> 5044
}
}
```
filter {
grok {
match => ["message", "%{DATA:event_timestamp} \(%{DATA:lmgrd}\) TIMESTAMP %{DATE:monthday}"]
tag_on_failure => ["message_data"]
}
mutate {
add_field => { "taskId" => "all" }
}
if "message_data" not in [tags] {
aggregate {
task_id => "%{taskId}"
code => "map['monthday'] = event.get('monthday')"
}
}
else {
aggregate {
task_id => "%{taskId}"
code => "if (map['monthday'] != nil) then event.set('monthday', map['monthday']) else event.set('monthday', (event.get('source').split('-')[3])) end"
}
grok {
match => ["message", "%{TIME:event_timestamp} \(%{DATA:lic_vendor_name}\) (?(OUT|IN|DENIED))\: \"%{DATA:lic_feature_name}\" %{DATA:user_name}@%{HOSTNAME:host_name}"] }
mutate { replace => ["event_timestamp", "%{monthday} %{event_timestamp}"] }
date { match => ["event_timestamp", "M/d/yyyy HH:mm:ss","M.d.yyyy HH:mm:ss","ISO8601"] target => "event_timestamp" }
}
```
}
output {
stdout {
codec =\> rubydebug
}
}
Before input line 3, the monthday should come from filename( 3rd field :5.11.2019) and after that monthday should get from the logfile itself from TIMESTAMP line ( 5/13/2019)
The output coming corresponding to input lines are
Line 1 : "event\_timestamp" =\> 2019-05-11T10:26:24.000Z, "monthday" =\> "5.11.2019"
Line 2: "event\_timestamp" =\> 2019-05-11T10:26:32.000Z "monthday" =\> "5.11.2019"
Line3 : "event\_timestamp" =\> "15:56:33" "monthday" =\> "5/13/2019"
Line4 : "event\_timestamp" =\> 2019-05-13T10:26:35.000Z "monthday" =\> "5/13/2019"
Line 5: "event\_timestamp" =\> 2019-05-11T10:26:37.000Z "monthday" =\> "5.11.2019"
Line 6: "event\_timestamp" =\> 2019-05-11T10:26:42.000Z, "monthday" =\> "5.11.2019"
Expected output should be
Line 1 : "event\_timestamp" =\> 2019-05-11T10:26:24.000Z, "monthday" =\> "5.11.2019"
Line 2: "event\_timestamp" =\> 2019-05-11T10:26:32.000Z "monthday" =\> "5.11.2019"
Line3 : "event\_timestamp" =\> "15:56:33" "monthday" =\> "5/13/2019"
Line4 : "event\_timestamp" =\> 2019-05-13T10:26:35.000Z "monthday" =\> "5/13/2019"
Line 5: "event\_timestamp" =\> 2019-05-11T10:26:37.000Z "monthday" =\> **"5/13/2019"**
Line 6: "event\_timestamp" =\> 2019-05-11T10:26:42.000Z, "monthday" =\> **"5/13/2019"**
Please suggest where I am doing it wrong?
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [June 11, 2019, 2:58pm UTC](https://discuss.elastic.co/t/aggregate-filter-not-working-correctly-iwth-logstash/185088/2 "2019-06-11T14:58:31Z")
> [@msk\_76](#):
>
> Please suggest where I am doing it wrong?
Does setting '--pipeline.batch.size 1' help?
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 9, 2019, 2:58pm UTC](https://discuss.elastic.co/t/aggregate-filter-not-working-correctly-iwth-logstash/185088/3 "2019-07-09T14:58:37Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.