How to use DATEDIFF with logstash jdbc_static filter

Hi,
I want to calculate the DATEDIFF under logstash and use it later under filter->jdbc_static ->local_lookups query for comparison.

when I tried to

  • Calculate DATEDIFF under filter->jdbc_static ->local_lookups , I get error as
    [WARN ] 2022-11-18 12:11:02.353 [[main]>worker1] lookup - Exception when executing Jdbc query {:lookup_id=>"local-servers", :exception=>"Java::JavaSql::SQLSyntaxErrorException: Column 'S' is either not in any table in the FROM list or appears within a join specification and is outside the scope of the join specification or appears in a HAVING clause and is not in the GROUP BY list. If this is a CREATE or ALTER TABLE statement then 'S' is not a column in the target table.", :backtrace=>["org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(org/apache/derby/impl/jdbc/S

Code

filter {
  ..............
  jdbc_static {
]
    local_lookups => [
      {
        id => "local-servers"
        query => "SELECT Id,timeToTestFilesAvailable FROM table01
                  WHERE Id = :testId and timeToTestFilesAvailable > DATEDIFF(s, :startedTime,:updatedTime)"
          parameters => {jobId => "[job_id]" updatedTime => "[updated]" startedTime => "[started]"}
      target => "jobIddetails"
      }
     ]

and

  • Calculate DATEDIFF in input plugin and use it under filter -> jdbc_static , I get Error - [WARN ] 2022-11-18 11:48:03.829 [[main]>worker1] lookup - Parameter field not found in event {:lookup_id=>"local-servers", :invalid_parameters=>["[inputFileWaitingTime]"]}

Code

input{
  jdbc {
    .......
    statement => "
      SELECT testId, updated, started, DATEDIFF(s, started,updated) inputFileWaitingTime
      FROM table02 with (nolock)
      WHERE
       updated > :sql_last_value
           AND status <> 0
            "
 } }

Can someone please assist me on how I can resolve the issue ?

Regrads,
Nivedita

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.