How to use DATEDIFF with logstash jdbc_static filter

I want to calculate the DATEDIFF under logstash and use it later under filter->jdbc_static ->local_lookups query for comparison.

when I tried to

  • Calculate DATEDIFF under filter->jdbc_static ->local_lookups , I get error as
    [WARN ] 2022-11-18 12:11:02.353 [[main]>worker1] lookup - Exception when executing Jdbc query {:lookup_id=>"local-servers", :exception=>"Java::JavaSql::SQLSyntaxErrorException: Column 'S' is either not in any table in the FROM list or appears within a join specification and is outside the scope of the join specification or appears in a HAVING clause and is not in the GROUP BY list. If this is a CREATE or ALTER TABLE statement then 'S' is not a column in the target table.", :backtrace=>["org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(org/apache/derby/impl/jdbc/S


filter {
  jdbc_static {
    local_lookups => [
        id => "local-servers"
        query => "SELECT Id,timeToTestFilesAvailable FROM table01
                  WHERE Id = :testId and timeToTestFilesAvailable > DATEDIFF(s, :startedTime,:updatedTime)"
          parameters => {jobId => "[job_id]" updatedTime => "[updated]" startedTime => "[started]"}
      target => "jobIddetails"


  • Calculate DATEDIFF in input plugin and use it under filter -> jdbc_static , I get Error - [WARN ] 2022-11-18 11:48:03.829 [[main]>worker1] lookup - Parameter field not found in event {:lookup_id=>"local-servers", :invalid_parameters=>["[inputFileWaitingTime]"]}


  jdbc {
    statement => "
      SELECT testId, updated, started, DATEDIFF(s, started,updated) inputFileWaitingTime
      FROM table02 with (nolock)
       updated > :sql_last_value
           AND status <> 0
 } }

Can someone please assist me on how I can resolve the issue ?


This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.