Logstash in docker input jdbc plugin not updating last_run_metadata_path

Elastic Stack Logstash
1

Hi I am trying to use the input jdbc plugin
jdbc { type => "logs" jdbc_driver_library => "/opt/logstash/driver/ojdbc6.jar" jdbc_driver_class => "Java::oracle.jdbc.driver.OracleDriver" jdbc_connection_string => "jdbc:oracle:thin:@//<host>:<port>/<database>" jdbc_user => "****" jdbc_password => "****" jdbc_fetch_size => 1000 schedule => "* * * * *" statement => "select CONNECTION_ID, IPADDRESS, START_DATE, ELAPSED_TIME_MS, GUI_EVENT_TYPE, GUI_EVENT_NAME, GUI_EVENT_PARAMS from xxxx where start_date>to_date(:sql_last_start, 'DD-MON-YY HH24:MI:SS')" clean_run => false record_last_run => true last_run_metadata_path => "/opt/logstash/lastrun/.logstash_jdbc_last_run" }
I have logstash running in a docker container. The metadata file is volume mapped as follows:

-v - /opt/logstash/lastrun/.logstash_jdbc_last_run:/opt/logstash/lastrun/.logstash_jdbc_last_run
However, I am not seeing logstash update the meta file with last run date. Not sure where the problem is. Any help would be great.
Thanks

2

Hello, you're wrong, you need to know this.The plugin will persist the sql_last_start parameter in the form of a metadata file stored in the configured last_run_metadata_path. Upon shutting down, this file will be updated with the current value of sql_last_start. Next time the pipeline starts up, this value will be updated by reading from the file. If clean_run is set to true, this value will be ignored and sql_last_start will be set to Jan 1, 1970, as if no query has ever been executed.

These are the official website of the document, I also want to update the configuration time, but I do not know, it is not possible to achieve

3

I don't see any value add in your answer by repeating the obvious. Yes, I checked the website documentation and understand what should happen. The question relates to why it is not. The problem was really with my docker volume mapping. I set it to -v - /opt/logstash/lastrun:/opt/logstash/lastrun. Docker processes files and directories differently.

4

Hi Sripri,

I'm encountring the same problem (getting an error like this :
No such file or directory - /var/www/.logstash_jdbc_last_run
Did you finally find an issue to this problem?

5

Hi @Djelouah_Laala @sripri
I'm having the same issue. Dont know how to have my dockerised logstash work with JDBC input and last_run_metadata_path...

Do you have any idea ?
Thanks a lot