# Run a filter that checks for values in previous records

**URL:** <https://discuss.elastic.co/t/run-a-filter-that-checks-for-values-in-previous-records/90663>\
**Category:** Logstash\
**Created:** [June 23, 2017, 3:33pm UTC](https://discuss.elastic.co/t/run-a-filter-that-checks-for-values-in-previous-records/90663 "2017-06-23T15:33:23Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Tejas\_Ghutukade](https://avatars.discourse-cdn.com/v4/letter/t/ba9def/32.png) [@Tejas\_Ghutukade](https://discuss.elastic.co/u/Tejas_Ghutukade)\
**Post date:** [June 23, 2017, 3:33pm UTC](https://discuss.elastic.co/t/run-a-filter-that-checks-for-values-in-previous-records/90663/1 "2017-06-23T15:33:23Z")

</div>

Hi,

i am passing a csv file to lagstach with records of timestamp and processing time.  
i want to create a fliter that checks if the current timestamp lies within the timestamp and processing time of any of the previous records and if condition true want to add a field with value 1 .

is it possible to do this ?

Thanks

---

<div class="post-metadata">

**Author:** ![Rory](https://avatars.discourse-cdn.com/v4/letter/r/a183cd/32.png) [@Rory](https://discuss.elastic.co/u/Rory)\
**Post date:** [June 23, 2017, 3:41pm UTC](https://discuss.elastic.co/t/run-a-filter-that-checks-for-values-in-previous-records/90663/2 "2017-06-23T15:41:14Z")

</div>

You should take a look at the logstash aggregate filter: [https://www.elastic.co/guide/en/logstash/current/plugins-filters-aggregate.html](https://www.elastic.co/guide/en/logstash/current/plugins-filters-aggregate.html)

One thing I would suggest is to sort the csv file by timestamp...

---

<div class="post-metadata">

**Author:** ![Tejas\_Ghutukade](https://avatars.discourse-cdn.com/v4/letter/t/ba9def/32.png) [@Tejas\_Ghutukade](https://discuss.elastic.co/u/Tejas_Ghutukade)\
**Post date:** [June 23, 2017, 3:43pm UTC](https://discuss.elastic.co/t/run-a-filter-that-checks-for-values-in-previous-records/90663/3 "2017-06-23T15:43:56Z")

</div>

> [@Rory](#):
>
> v file by ti

yes its already sorted.I'll have a look at the aggregate filter.

Thanks

---

<div class="post-metadata">

**Author:** ![Tejas\_Ghutukade](https://avatars.discourse-cdn.com/v4/letter/t/ba9def/32.png) [@Tejas\_Ghutukade](https://discuss.elastic.co/u/Tejas_Ghutukade)\
**Post date:** [June 23, 2017, 8:20pm UTC](https://discuss.elastic.co/t/run-a-filter-that-checks-for-values-in-previous-records/90663/4 "2017-06-23T20:20:39Z")

</div>

@Rory  
I tried aggregate filter. here's my filter code

```
aggregate {
	task_id => "%{TRANSACTIONIDGLOBAL}"
	code => " 	
	map['tr_start'] ||= 0;
	map['tr_end'] ||= 0;
	event.set('cc' , event.get('starttime') < map['tr_end']);			
	map['tr_start'] = event.get('starttime');
	map['tr_end'] = event.get('endtime');
	"
	push_map_as_event_on_timeout => true
	timeout => 120
}	

```

so here m trying to check if for the current record the starttime is whether less than the previous record endtime.  
but in output its checking the start time with the endtime of the current record itself.  
at the comparision level istead of checking with the previous record its checking with itself.

Thanks

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 21, 2017, 8:21pm UTC](https://discuss.elastic.co/t/run-a-filter-that-checks-for-values-in-previous-records/90663/5 "2017-07-21T20:21:18Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.