# Drop events older than three days
**URL:** https://discuss.elastic.co/t/drop-events-older-than-three-days/83423
**Category:** Logstash
**Created:** [April 24, 2017, 2:23pm UTC](https://discuss.elastic.co/t/drop-events-older-than-three-days/83423 "2017-04-24T14:23:28Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@seth.yes](https://discuss.elastic.co/u/seth.yes)
#### Post date: [April 24, 2017, 2:23pm UTC](https://discuss.elastic.co/t/drop-events-older-than-three-days/83423/1 "2017-04-24T14:23:28Z")
I'm using Logstash v5.3 on Ubuntu 14.04. I'm looking to drop events sent to logstash that have a timestamp which is three days old or more.
I'm parsing the date off the events and using the 'date' plugin to index of the date of the event creation (vs the default of the date ingested).
I want to capture as much of this data as I can, however I'm implementing a hot/warm architecture and want to just delete data older than three days so that a new index isn't created on my hot nodes.
Does anyone know of a good way to do this?
**_The code block I want to implement this on:_**
```auto
if [ts_request] {
mutate {
convert => ["ts_request", "string"]
}
date {
match => ["ts_request", "dd/MMM/YYYY:HH:mm:ss Z"]
target => "@timestamp"
add_field => { "date_parser" => "indexed on date by %{LogstashServer}" }
}
}
```
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [April 24, 2017, 2:35pm UTC](https://discuss.elastic.co/t/drop-events-older-than-three-days/83423/2 "2017-04-24T14:35:16Z")
Maybe [the age filter plugin](https://www.elastic.co/guide/en/logstash/current/plugins-filters-age.html) might be useful?
---
### Author:  [@seth.yes](https://discuss.elastic.co/u/seth.yes)
#### Post date: [April 24, 2017, 2:38pm UTC](https://discuss.elastic.co/t/drop-events-older-than-three-days/83423/3 "2017-04-24T14:38:39Z")
> [@Christian\_Dahlqvist](#):
>
> the age filter plugin
This looks like it'll work. I'll update this thread with any nuances or uncharacteristic operation of that plugin, but from looking at the functionality, that's just what I need.
---
### Author:  [@seth.yes](https://discuss.elastic.co/u/seth.yes)
#### Post date: [April 25, 2017, 5:35pm UTC](https://discuss.elastic.co/t/drop-events-older-than-three-days/83423/4 "2017-04-25T17:35:26Z")
The 'age' filter is not operating in the manner I was hoping -- it's apparently measuring the duration between when the data is sent from Filebeat and when it hits my Logstash filter.
I've implemented the filter in the following manner:
```auto
filter {
age {}
if [@metadata][age] > 86400 {
drop {}
}
else {
mutate {
add_field => {"index_latency_es" => "%{[@metadata][age]}" }
}
}
}
```
Which does give me an age, e.g.
(This latency is larger than usual, tweaking a few things at the moment which caused this spike in latency)
However, I'm seeing indices that are days-old still being populated.
###Does anyone know how the [Age Filter Plugin](https://www.elastic.co/guide/en/logstash/current/plugins-filters-age.html) operates?
###How could I do a comparison of the current day to the day being indexed?
####Should I approach this issue in a different way?
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [April 25, 2017, 5:47pm UTC](https://discuss.elastic.co/t/drop-events-older-than-three-days/83423/5 "2017-04-25T17:47:43Z")
Based on the description I believe it operates on the `@timestamp` field. This will be the timestamp the event was read by Filebeat if you have not used a date filter to parse the log timestamp into `@timestamp` before you call the age filter. If you parse you log timestamp first I think it should behave the way you expect.
---
### Author:  [@seth.yes](https://discuss.elastic.co/u/seth.yes)
#### Post date: [April 25, 2017, 7:16pm UTC](https://discuss.elastic.co/t/drop-events-older-than-three-days/83423/6 "2017-04-25T19:16:08Z")
@Christian_Dahlqvist you were completely correct. I moved this filter to the end of my config, just before the output and it works just as expected.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 23, 2017, 7:18pm UTC](https://discuss.elastic.co/t/drop-events-older-than-three-days/83423/7 "2017-05-23T19:18:01Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.