# Elapsed filter is not working

**URL:** <https://discuss.elastic.co/t/elapsed-filter-is-not-working/91530>\
**Category:** Logstash\
**Created:** [July 1, 2017, 10:49pm UTC](https://discuss.elastic.co/t/elapsed-filter-is-not-working/91530 "2017-07-01T22:49:27Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![ibrahimsharaf](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ibrahimsharaf/32/17304_2.png) [@ibrahimsharaf](https://discuss.elastic.co/u/ibrahimsharaf)\
**Post date:** [July 1, 2017, 10:49pm UTC](https://discuss.elastic.co/t/elapsed-filter-is-not-working/91530/1 "2017-07-01T22:49:27Z")

</div>

Hello, I am trying to use Elapsed filter to get the duration between two timestamps, here's a snap of my logfile:

```
2017-01-01 07:53:44 [utils.py] WARNING: enable_proxy must have atleast one http
2017-01-01 07:53:45 [provider.py] DEBUG: Using access key found in environment variable.
2017-01-01 07:53:50 [engine.py] INFO: Spider opened
2017-01-01 07:54:01 [logstats.py] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2017-01-01 07:55:44 [monitor_utils.py] INFO: Getting the latest iteration for merchant ariika

```

I am trying to get the duration between the first and last lines.

here's my logstash configuration:

```
input {
    tcp {
        port => 5000
        codec => multiline {
            pattern => "^%{TIMESTAMP_ISO8601} "
            negate => true
            what => previous
        }
    }
}

filter {

		grok{
    			match => ["message", "%{TIMESTAMP_ISO8601} %{NOTSPACE} WARNING: enable_proxy must %{GREEDYDATA:task_id}"]
  				add_tag => ["taskStarted"]
  		}

  		grok{
  				match => ["message", "%{TIMESTAMP_ISO8601} %{NOTSPACE} INFO: Getting the latest iteration for %{GREEDYDATA:task_id}"]
  				add_tag => ["taskTerminated"]
		}

		grok{
				match => ["message", "%{DATE_EU:timestamp}"]
			}
			
		date{
		    	match => ["timestamp", "yy-MM-dd"]
		   		target => "@timestamp"
			}

		elapsed{
    			start_tag => "taskStarted"
    			end_tag => "taskTerminated"
    			unique_id_field => "task_id"
    			new_event_on_match => true
  		}
	}

output {
	if "_grokparsefailure" not in [tags]{
		stdout {
		codec => rubydebug
		}
         } 
 }

```

my stdout only contains those lines:

```
logstash_1 | [2017-06-20T15:12:48,787][INFO][logstash.filters.elapsed] Elapsed, 'start event' received {:start_tag=>"taskStarted", :unique_id_field=>"task_id"}
logstash_1 | [2017-06-20T15:12:48,848][INFO][logstash.filters.elapsed] Elapsed, 'end event' received {:end_tag=>"taskTerminated", :unique_id_field=>"task_id"}
```

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [July 2, 2017, 12:36am UTC](https://discuss.elastic.co/t/elapsed-filter-is-not-working/91530/2 "2017-07-02T00:36:13Z")

</div>

There's no `task_id` in the logs to calculate on that I can see?

---

<div class="post-metadata">

**Author:** ![ibrahimsharaf](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ibrahimsharaf/32/17304_2.png) [@ibrahimsharaf](https://discuss.elastic.co/u/ibrahimsharaf)\
**Post date:** [July 2, 2017, 1:25am UTC](https://discuss.elastic.co/t/elapsed-filter-is-not-working/91530/3 "2017-07-02T01:25:24Z")

</div>

@warkolm So won't I be able to use elapsed filter with my logs in this form? If not, do you have other options in mind to calculate the duration instead of elapsed filter?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [July 2, 2017, 9:08pm UTC](https://discuss.elastic.co/t/elapsed-filter-is-not-working/91530/4 "2017-07-02T21:08:09Z")

</div>

It's hard because there is no identifier to link these all together.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 30, 2017, 9:08pm UTC](https://discuss.elastic.co/t/elapsed-filter-is-not-working/91530/5 "2017-07-30T21:08:32Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.