# Date filter plugin issue

**URL:** <https://discuss.elastic.co/t/date-filter-plugin-issue/138208>\
**Category:** Logstash\
**Created:** [July 2, 2018, 12:28pm UTC](https://discuss.elastic.co/t/date-filter-plugin-issue/138208 "2018-07-02T12:28:26Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![dyl](https://avatars.discourse-cdn.com/v4/letter/d/5fc32e/32.png) [@dyl](https://discuss.elastic.co/u/dyl)\
**Post date:** [July 2, 2018, 12:28pm UTC](https://discuss.elastic.co/t/date-filter-plugin-issue/138208/1 "2018-07-02T12:28:26Z")

</div>

Hey,

I try to parse a duration field thanks to "date" filter plugin but i'm unable to do this correctly \>\<

My log : 0h 41' 47" 175ms

I tried :

`H'h' MM\' ss'"' SSS'ms'`

But it doesn't work..I am a little confused with the ' and " etc...

Can somebody help me to find the good date filter ?

Some exemple of my logs :

0h 1' 24" 811ms  
0h 1' 51" 430ms  
0h 2' 0" 73ms  
0h 41' 2" 493ms  
.  
.

Thx u !

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [July 2, 2018, 12:36pm UTC](https://discuss.elastic.co/t/date-filter-plugin-issue/138208/2 "2018-07-02T12:36:09Z")

</div>

The date filter doesn't parse durations.

---

<div class="post-metadata">

**Author:** ![dyl](https://avatars.discourse-cdn.com/v4/letter/d/5fc32e/32.png) [@dyl](https://discuss.elastic.co/u/dyl)\
**Post date:** [July 2, 2018, 12:38pm UTC](https://discuss.elastic.co/t/date-filter-plugin-issue/138208/3 "2018-07-02T12:38:03Z")

</div>

Hm ok, so how can I do to handle with duration field ?

I want to do that to do a query which search all event where this field is \> 2 minutes for exemple. Is it possible to do this kind of "where" query ?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [July 2, 2018, 1:39pm UTC](https://discuss.elastic.co/t/date-filter-plugin-issue/138208/4 "2018-07-02T13:39:48Z")

</div>

The easiest option is probably to write some Ruby code in a ruby filter.

---

<div class="post-metadata">

**Author:** ![dyl](https://avatars.discourse-cdn.com/v4/letter/d/5fc32e/32.png) [@dyl](https://discuss.elastic.co/u/dyl)\
**Post date:** [July 2, 2018, 1:52pm UTC](https://discuss.elastic.co/t/date-filter-plugin-issue/138208/5 "2018-07-02T13:52:29Z")

</div>

I don't know ruby :s

Do you have any ideas to help me?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [July 2, 2018, 2:04pm UTC](https://discuss.elastic.co/t/date-filter-plugin-issue/138208/6 "2018-07-02T14:04:41Z")

</div>

Sorry, I don't have time to give detailed guidance.

---

<div class="post-metadata">

**Author:** ![dyl](https://avatars.discourse-cdn.com/v4/letter/d/5fc32e/32.png) [@dyl](https://discuss.elastic.co/u/dyl)\
**Post date:** [July 2, 2018, 2:05pm UTC](https://discuss.elastic.co/t/date-filter-plugin-issue/138208/7 "2018-07-02T14:05:23Z")

</div>

Maybe @Badger can help me, will see..

But thx for help !

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 2, 2018, 3:10pm UTC](https://discuss.elastic.co/t/date-filter-plugin-issue/138208/8 "2018-07-02T15:10:34Z")

</div>

You could parse it using grok. If you really do just want to test for the number of minutes being greater than two then you don't even need to combine the pieces.

```
grok { match => ["message", "%{NUMBER:h:int}h %{NUMBER:m:int}' %{NUMBER:s:int}\" %{NUMBER:ms:int}ms"] }
```

---

<div class="post-metadata">

**Author:** ![dyl](https://avatars.discourse-cdn.com/v4/letter/d/5fc32e/32.png) [@dyl](https://discuss.elastic.co/u/dyl)\
**Post date:** [July 3, 2018, 6:06am UTC](https://discuss.elastic.co/t/date-filter-plugin-issue/138208/9 "2018-07-03T06:06:05Z")

</div>

> [@Badger](#):
>
> %{NUMBER:h:int}h %{NUMBER:m:int}' %{NUMBER:s:int}" %{NUMBER:ms:int}ms

Ok and with this grok i would like to do : all my events where "m" \> "2". Can I do that ?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [July 3, 2018, 6:41am UTC](https://discuss.elastic.co/t/date-filter-plugin-issue/138208/10 "2018-07-03T06:41:36Z")

</div>

```
if [m] > 2 {
    ...
}

```

Now of course this won't catch a duration like "1h 0m 0s" which is why I suggested a Ruby-based solution that produces a single integer representing the total number of milliseconds (or whatever unit of resolution is desired).

---

<div class="post-metadata">

**Author:** ![dyl](https://avatars.discourse-cdn.com/v4/letter/d/5fc32e/32.png) [@dyl](https://discuss.elastic.co/u/dyl)\
**Post date:** [July 3, 2018, 11:51am UTC](https://discuss.elastic.co/t/date-filter-plugin-issue/138208/11 "2018-07-03T11:51:50Z")

</div>

So ok I can do that into my logstash pipeline,

but me I would like to that into Kibana, for exemple to have the name of my event where m \> 2

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [July 3, 2018, 1:30pm UTC](https://discuss.elastic.co/t/date-filter-plugin-issue/138208/12 "2018-07-03T13:30:20Z")

</div>

The query string `duration:[2000000 TO *]` finds all events where the `duration` field is 2000000 or greater.

---

<div class="post-metadata">

**Author:** ![dyl](https://avatars.discourse-cdn.com/v4/letter/d/5fc32e/32.png) [@dyl](https://discuss.elastic.co/u/dyl)\
**Post date:** [July 4, 2018, 1:19pm UTC](https://discuss.elastic.co/t/date-filter-plugin-issue/138208/13 "2018-07-04T13:19:55Z")

</div>

Ok thx so I have 3 format of duration log :

```
0h 1' 1" 646ms
2 mn 35 s 288 ms
24 s 515 ms

```

I did my grok, it work well :

```
 grok
    {
      match => { "DESCRIPTION" => ["%{NUMBER:HEURE:int}h %{NUMBER:MINUTE:int}' %{NUMBER:SECONDE:int}\" %{NUMBER:MILLISECONDE:int}ms","%{NUMBER:MINUTE:int} mn %{NUMBER:SECONDE:int} s %{NUMBER:MILLISECONDE:int} ms","%{NUMBER:SECONDE:int} s %{NUMBER:MILLISECONDE:int} ms"] }
    }

```

But I save all the fields : (for further use)

```
ruby { code => "@@save_the_heure = event.get('HEURE')" }
ruby { code => "@@save_the_minute = event.get('MINUTE')" }
ruby { code => "@@save_the_seconde = event.get('SECONDE')" }
ruby { code => "@@save_the_milliseconde = event.get('MILLISECONDE')" }

```

And sometime (for exemple my second log line exemple), my field "HEURE" is empty.. logic, so I want to add 0 instead of nil and then to convert in integer but it doesn't work like that :

```
if !([HEURE]) 
{
  mutate
  {
    add_field => { "HEURE" => "0" }
    convert => { "HEURE" => "integer" }
  }	
}

```

Have you got another solution ?

EDIT : Solution :

```
if !([HEURE]) 
{
  mutate { add_field => { "HEURE" => 0 } }
  mutate { convert => ["HEURE","integer"] }
}
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 1, 2018, 1:20pm UTC](https://discuss.elastic.co/t/date-filter-plugin-issue/138208/14 "2018-08-01T13:20:00Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.