# Logstash date filter question

**URL:** https://discuss.elastic.co/t/logstash-date-filter-question/33663
**Category:** Logstash
**Created:** [November 3, 2015, 6:45pm UTC](https://discuss.elastic.co/t/logstash-date-filter-question/33663 "2015-11-03T18:45:52Z")
**Posts on this page:** 13
**Page:** 1

<div class="post-metadata">

### Author: ![CraigFoote](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/craigfoote/32/4571_2.png) [@CraigFoote](https://discuss.elastic.co/u/CraigFoote)
#### Post date: [November 3, 2015, 6:45pm UTC](https://discuss.elastic.co/t/logstash-date-filter-question/33663/1 "2015-11-03T18:45:52Z")

</div>

I'm on the 2.0 stack and am using the JDBC input plugin but a couple DB fields that should be dates are coming in as text, "event\_date" and "event\_timestamp". I'm trying to use the date filter to convert them to dates but can't get it to work. Any help greatly appreciated.

```
input{
    jdbc{
        ...
    }
}
filter{
    date{
        match => ["event_date", "YYYY-MM-dd HH:mm:ss Z"]
        target => "event_date"    
}
output{
    elasticsearch{
        ...
    }
}

```

Configtest comes back ok but running it I get:

> Failed parsing date from field {:field=\>"event\_date", :value=\>2015-11-02 02:24:56 -500, :exception=\>cannot convert instance of class org.jruby.RubyTime to class java.lang.String", :config\_parsers=\>"YYYY-MM-dd HH:mm:ss Z", :config\_locale=\>"default=en\_US", :level=\> :warn}

To me this suggests the fields are aleady dates but the Discover tab in Kibana reports them as "t".

Also, am I using the "target" option correctly? I want to reuse the same field but as a date type, or should I be creating another field "event\_date\_as\_date"? I tried commenting that line out to target @timestamp but got the same error. I have 2 date fields seemingly indexed as text so I can't target @timestamp anyway.

---

<div class="post-metadata">

### Author: ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)
#### Post date: [November 3, 2015, 7:01pm UTC](https://discuss.elastic.co/t/logstash-date-filter-question/33663/2 "2015-11-03T19:01:20Z")

</div>

> [@CraigFoote](#):
>
> :exception=\>cannot convert instance of class org.jruby.RubyTime to class java.lang.String"

You don't need to use the date filter as the field is already a "RubyTime" object.

---

<div class="post-metadata">

### Author: ![CraigFoote](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/craigfoote/32/4571_2.png) [@CraigFoote](https://discuss.elastic.co/u/CraigFoote)
#### Post date: [November 4, 2015, 1:54am UTC](https://discuss.elastic.co/t/logstash-date-filter-question/33663/3 "2015-11-04T01:54:35Z")

</div>

Thanks for responding Aaron. So why does event\_date show up as text in Kibana? And I'm not able to use it in time-based graphs. Also it turns out the 2nd field I mentioned, "event\_timestamp", is superfluous and I do need to use "event\_date" as the @timestamp for the event. How do I do that? I need to reassign the value of @timestamp to the value of event\_date. Don't I need to use the date filter for that?

---

<div class="post-metadata">

### Author: ![CraigFoote](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/craigfoote/32/4571_2.png) [@CraigFoote](https://discuss.elastic.co/u/CraigFoote)
#### Post date: [November 4, 2015, 3:04pm UTC](https://discuss.elastic.co/t/logstash-date-filter-question/33663/4 "2015-11-04T15:04:40Z")

</div>

I tried changing the date filter to a ruby filter:

```
ruby{
    code => "event['@timestamp'] = event['event_date']"
}

```

...but got this error:

> Ruby exception occurred: The field '@timestamp' must be a (Logstash:Timestamp, not a Time (2015-11-03 02:30:43 -500) {:level=\>:error}

I then tried a mutate filter:

```
mutate{
    replace => { "@timestamp" => "event_date" }
}

```

... but got this error:

> TypeError: The field '@timestamp' must be a (Logstash::Timestamp, not a String (event\_date)

All I need to do is use the event\_date as @timestamp, what am I missing?

---

<div class="post-metadata">

### Author: ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)
#### Post date: [November 4, 2015, 3:11pm UTC](https://discuss.elastic.co/t/logstash-date-filter-question/33663/5 "2015-11-04T15:11:31Z")

</div>

These are excellent questions. It seems that something unusual may be going on in the JDBC input with regards to the date field. I will consult with my colleagues on how we can address that part.

Meanwhile, there's probably a ruby code block which will convert the `Time()` value into a `Logstash::Timestamp`. I'll do some digging and see if I can't get a working conversion.

---

<div class="post-metadata">

### Author: ![CraigFoote](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/craigfoote/32/4571_2.png) [@CraigFoote](https://discuss.elastic.co/u/CraigFoote)
#### Post date: [November 4, 2015, 3:23pm UTC](https://discuss.elastic.co/t/logstash-date-filter-question/33663/6 "2015-11-04T15:23:34Z")

</div>

Thanks again Aaron, I really appreciate this.

---

<div class="post-metadata">

### Author: ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)
#### Post date: [November 4, 2015, 4:20pm UTC](https://discuss.elastic.co/t/logstash-date-filter-question/33663/7 "2015-11-04T16:20:15Z")

</div>

Still working on a temporary workaround, but [here's the issue I raised](https://github.com/logstash-plugins/logstash-input-jdbc/issues/77).

---

<div class="post-metadata">

### Author: ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)
#### Post date: [November 4, 2015, 5:59pm UTC](https://discuss.elastic.co/t/logstash-date-filter-question/33663/8 "2015-11-04T17:59:05Z")

</div>

@CraigFoote, this should be a temporary workaround. We'll put this into the JDBC input plugin soon.

Put this in your filter block to convert the `RubyTime` object to a `LogStash::Timestamp` object:

```
ruby {
  code => "event['event_date'] = LogStash::Timestamp.new(event['event_date'])"
}

```

You may want to wrap this in a conditional to only have it work on the proper events.

---

<div class="post-metadata">

### Author: ![CraigFoote](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/craigfoote/32/4571_2.png) [@CraigFoote](https://discuss.elastic.co/u/CraigFoote)
#### Post date: [November 4, 2015, 7:56pm UTC](https://discuss.elastic.co/t/logstash-date-filter-question/33663/9 "2015-11-04T19:56:46Z")

</div>

Hi Aaron, thanks for the fix. I'm sure it'll work but I can't test it right now. We just undertook the task to install Shield, Watcher and Marvel so everything's down right now. I did run the JDBC source with the "event\_date" and "event\_timestamp" fields on our pre-2.0 stack cluster and they index as dates. This bug appears to have been introduced with 2.0.

---

<div class="post-metadata">

### Author: ![CraigFoote](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/craigfoote/32/4571_2.png) [@CraigFoote](https://discuss.elastic.co/u/CraigFoote)
#### Post date: [November 6, 2015, 1:56pm UTC](https://discuss.elastic.co/t/logstash-date-filter-question/33663/10 "2015-11-06T13:56:54Z")

</div>

Bad news Aaron, I'm getting an error with your code:

```
Ruby exception occurred: uninitialized constant LogStash::Filters::Ruby::Logstash {:level =>:error, :file=>"logstash/filters/ruby.rb", :line=>"41", :method=>"filter"}

```

I'm not a Ruby developer so any more help you could provide would be greatly appreciated.

---

<div class="post-metadata">

### Author: ![CraigFoote](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/craigfoote/32/4571_2.png) [@CraigFoote](https://discuss.elastic.co/u/CraigFoote)
#### Post date: [November 9, 2015, 2:25pm UTC](https://discuss.elastic.co/t/logstash-date-filter-question/33663/11 "2015-11-09T14:25:07Z")

</div>

bump bump

---

<div class="post-metadata">

### Author: ![CraigFoote](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/craigfoote/32/4571_2.png) [@CraigFoote](https://discuss.elastic.co/u/CraigFoote)
#### Post date: [November 9, 2015, 4:23pm UTC](https://discuss.elastic.co/t/logstash-date-filter-question/33663/12 "2015-11-09T16:23:59Z")

</div>

My apologies, I transcribed "LogStash::Timestamp.new" as "Logstash::Timestamp.new". When I capitalized the "S" your code worked. Thanks again.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 5:23am UTC](https://discuss.elastic.co/t/logstash-date-filter-question/33663/13 "2017-07-06T05:23:23Z")

</div>


