# Logstash - filter and drop lines that record\_date is older than 30mins of current timestamp?
**URL:** https://discuss.elastic.co/t/logstash-filter-and-drop-lines-that-record-date-is-older-than-30mins-of-current-timestamp/51975
**Category:** Logstash
**Created:** [June 6, 2016, 3:32pm UTC](https://discuss.elastic.co/t/logstash-filter-and-drop-lines-that-record-date-is-older-than-30mins-of-current-timestamp/51975 "2016-06-06T15:32:35Z")
**Posts on this page:** 5
**Page:** 1
### Author:  [@mt.michael](https://discuss.elastic.co/u/mt.michael)
#### Post date: [June 6, 2016, 3:32pm UTC](https://discuss.elastic.co/t/logstash-filter-and-drop-lines-that-record-date-is-older-than-30mins-of-current-timestamp/51975/1 "2016-06-06T15:32:35Z")
Hi. I have issue where I'd duplicate data send to the target through filebeat \> logstash \> target.
I couldn't figure what was the issue on the server as it was terminated in aws autoscale so I'm thinking of having a filter that will compare a field, record\_date, in the event to the @timestamp and if it's older than 30 mins then drop the line or ignore it so it won't send to target.
Is this something needs to be done with ruby code? Or as simple as through with the "if" ?
Thanks.
---
### Author:  [@purbon](https://discuss.elastic.co/u/purbon)
#### Post date: [June 6, 2016, 3:50pm UTC](https://discuss.elastic.co/t/logstash-filter-and-drop-lines-that-record-date-is-older-than-30mins-of-current-timestamp/51975/2 "2016-06-06T15:50:13Z")
Hi,
can you share your config, logstash version and operating system? also having more info on your architecture would be nice to know where the issue might be.
---
### Author:  [@mt.michael](https://discuss.elastic.co/u/mt.michael)
#### Post date: [June 6, 2016, 4:11pm UTC](https://discuss.elastic.co/t/logstash-filter-and-drop-lines-that-record-date-is-older-than-30mins-of-current-timestamp/51975/3 "2016-06-06T16:11:23Z")
Hi. My logstash version is 2.1.1 and it's running on Ubuntu 14.04.4 LTS.
Here's my logstash config:
input {
beats {
port =\> 5044
}
}
filter {
json {
source =\> "message"
}
mutate {
add\_field =\> { "logtype" =\> "test" }
remove\_field =\> "message"
remove\_field =\> "count"
remove\_field =\> "fields"
remove\_field =\> "fileinfo"
remove\_field =\> "input\_type"
remove\_field =\> "line"
remove\_field =\> "offset"
remove\_field =\> "shipper"
}
}
output {
if [function] == "log" {
```
rabbitmq {
{...}
}
```
}
}
My filebeat data file: {"function":"log","newRecordData":{"record\_time":"2016-06-06 11:53:07","name": "test"}}
I was thinking if there's a way to see the "record\_time" and see if it's older than 30mins then ignore the line.
I see example from others using filter along with ruby code but their example are just add tag and also their ruby code is not similar case. Maybe what I'm trying to do is as simple as inside the "if" in the output.
I'm wondering if there's something like this?
if [function] == "log" && event['newRecordData']['record\_time'] \< [current\_time - 30mins]
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [June 10, 2016, 5:33am UTC](https://discuss.elastic.co/t/logstash-filter-and-drop-lines-that-record-date-is-older-than-30mins-of-current-timestamp/51975/4 "2016-06-10T05:33:57Z")
You need a ruby filter for the timestamp math. Parse the `record_time` field to a DateTime object and subtract that from DateTime.now. I believe the result is the time difference expressed as fractions of a day, so multiple that by 24\*60 and you'll get the difference in minutes.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 4:53am UTC](https://discuss.elastic.co/t/logstash-filter-and-drop-lines-that-record-date-is-older-than-30mins-of-current-timestamp/51975/5 "2017-07-06T04:53:47Z")