Logstash JDBC Plugin - optimal way to fetch new incoming data


(Samnik60) #1

Hi guys,

Aim : To take data from a oracle table at regular interval without overlapping data ( i.e data from first run doesnt show up in next run ) to a target .

problem : How to query the table for the above use case if it doesnt have an insertion time column.

Probable solution if this feature exists : since we dont have the insertion time column , if can
store any other meta data from the query, apart from :sql_last_run, say for example the last records primary key , we can solve this problem. Does logstash jdbc plugin have such feature ?

If not then this plugin is useful only if we can add an insertion time column in the table.

Thanks,
Sam.


(system) #2