Issue with selecting date fields from the `jdbc_streaming` `filter` plugin

[nkumarcc (Nitin Kumar)]

I have a nested field in my Elasticsearch mapping which holds an array of objects. We wanted to add this array to records via a jdbc_streaming filter. However, 2 of the fields we query for are timestamp fields, which cause a conversion error as Ruby tries to turn the dates into RubyDate values.

I tried casting the dates to varchar in the query, then converting them to dates after via a ruby filter. This just caused the same issue

We were able to solve this using an aggregate filter instead, but it was much less concise and this felt like an edge case that could be addressed. Can we select date and time fields from jdbc_streaming somehow?

filter {
  jdbc_streaming {
    ...
    statement => "select timestampField1, timestampField2 from table WHERE id = ?"
    ...
  }
}