Logstash JDBC Input Complete Action

[Jeff_Bursik]

This has been asked in other threads however a resolution or a solution doesn't appear to have ever been posted so I'd like to ask agin:

Is it possible to have a post action once Logstash has completed the stream/output of the data? For example, I have a JDBC input and ES output that runs every 5 minutes. Upon the completion of the 5 minute task, I'd like to call an API to post data.

If Logstash doesn't contain this function, is there another recommend way? I have considered having logstash running on both the Logstash file (looking for the print out of the SQL statement) and JDBC but that doesn't seem very efficient.

Any help would be great!

[magnusbaeck (Magnus Bäck)]

Is it possible to have a post action once Logstash has completed the stream/output of the data?

No. There's nothing built-in for this and I can't come up with a workaround that's clearly better than what you're suggesting (reading the log).

Can plugin provide "EOF" as tag to signal End of SQL data?

[Jeff_Bursik]

Thanks @magnusbaeck.

After some trial and error, it dawned on me that I can modify the sql statement to add a final row. A Union and ensure the final row is always at the bottom of the result set. Appears to be working perfectly for me, incase anyone comes across this.

input {
 jdbc {
   statement => 
          "SELECT
                'RUNNING' as logstashstatus
              FROM TABLE
           union
              SELECT
                 'ZCOMPLETE' as logstashstatus
              FROM DUAL
           order by logstashstatus"
 }
}

filter {
 .... 
}


output {

  stdout {}

  if [logstashstatus] != "ZCOMPLETE" {
     .... output
  }
  
  # For JDBC Input Streaming
  if [logstashstatus] == "ZCOMPLETE" {
    # Delete the Status file before writing to it so
    exec {
      command => "echo DONE!!!!!!!!!!! "
    }
  }
}