# How to process logs from certain period of time? (I use elasticsearch input plugin)
**URL:** https://discuss.elastic.co/t/how-to-process-logs-from-certain-period-of-time-i-use-elasticsearch-input-plugin/278702
**Category:** Logstash
**Created:** [July 14, 2021, 6:12pm UTC](https://discuss.elastic.co/t/how-to-process-logs-from-certain-period-of-time-i-use-elasticsearch-input-plugin/278702 "2021-07-14T18:12:14Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@John\_Smith1](https://discuss.elastic.co/u/John_Smith1)
#### Post date: [July 14, 2021, 6:12pm UTC](https://discuss.elastic.co/t/how-to-process-logs-from-certain-period-of-time-i-use-elasticsearch-input-plugin/278702/1 "2021-07-14T18:12:15Z")
In pipeline I receive logs from server with elasticsearch input plugin and I want to process not all logs, but for example only last 2 months, or, preferably from certain date to certain date.
When I receive logs there is already separate field @timestamp, so I tried to use this
```auto
input {
elasticsearch {
host =>
index =>
...and so on...
}
}
filter {
if [@timestamp] >= "May 31, 2021 @ 23:59:56.672" {
# grok and other stuff.
}
}
```
but this doesn't work
I know about ignore\_older in file input plugin, but that doesn't work for me.
Couldn't find anything with google either.
If you could give me some advice of suggestions that would be awesome.
Thank you.
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [July 14, 2021, 6:21pm UTC](https://discuss.elastic.co/t/how-to-process-logs-from-certain-period-of-time-i-use-elasticsearch-input-plugin/278702/2 "2021-07-14T18:21:26Z")
There is an example of testing the age of an event [here](https://discuss.elastic.co/t/logstash-filter-basted-on-log-file-age/241350/2).
---
### Author:  [@John\_Smith1](https://discuss.elastic.co/u/John_Smith1)
#### Post date: [July 14, 2021, 7:19pm UTC](https://discuss.elastic.co/t/how-to-process-logs-from-certain-period-of-time-i-use-elasticsearch-input-plugin/278702/3 "2021-07-14T19:19:01Z")
Thank you, I tried but pipeline still process all logs, probably I doing something wrong.
I tried to parse only last 2 days logs with this:
```auto
input {
elasticsearch {
host =>
index =>
...and so on...
}
}
filter {
ruby { code => 'event.set("[@metadata][age]", Time.now.to_f - event.timestamp.to_f)' }
mutate { convert => { "[@metadata][age]" => "integer" } }
if 86400 < [@metadata][age] {
# grok and other conditions.
```
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [July 14, 2021, 7:44pm UTC](https://discuss.elastic.co/t/how-to-process-logs-from-certain-period-of-time-i-use-elasticsearch-input-plugin/278702/4 "2021-07-14T19:44:28Z")
> [@John\_Smith1](#):
>
> ```auto
> if 86400 < [@metadata][age] {
> # grok and other conditions.
>
> ```
The events will still go through the pipeline unless you drop them. That condition means event will only go through the grok filter if they are more than one day old. Perhaps what you want is
```
if [@metadata][age] > 172400 { drop {} }
```
---
### Author:  [@John\_Smith1](https://discuss.elastic.co/u/John_Smith1)
#### Post date: [July 15, 2021, 7:28am UTC](https://discuss.elastic.co/t/how-to-process-logs-from-certain-period-of-time-i-use-elasticsearch-input-plugin/278702/5 "2021-07-15T07:28:32Z")
Thank you, it's works. I thought maybe if I skip some logs, logstash "jump" to logs that I want and whole process will be faster, but looks like it's still more or less same speed,
like even if it drop logs from certain period, it still check it and it take a lot of time if there are a lot of logs.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 12, 2021, 7:29am UTC](https://discuss.elastic.co/t/how-to-process-logs-from-certain-period-of-time-i-use-elasticsearch-input-plugin/278702/6 "2021-08-12T07:29:00Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.