I'm currently looking into JDBC Input plugin for Oracle and I have one concern. Assuming my pipeline is timestamp based with the following format: yyyy-mm-dd hh:mm:ss. Let's say that there are two event that occurred at the same time and Logstash run was started at the exact same time when those two events occured. So the timeline woud look like this:
2019-04-04 09:40:02 -> DB entry 1
2019-04-04 09:40:02 -> Logstash starts
2019-04-04 09:40:02 -> DB entry 2
2019-04-04 09:40:05 -> DB entry 3
2019-04-04 09:40:06 -> Logstash starts
Since my pipeline is timestamp based, i have the following config snippet:
where timestamp > :sql_last_value
I assume that after first Logstash run at 2019-04-04 09:40:02 .logstash_jdbc_last_run will contain 2019-04-04 09:40:02 and store it as a reference. It means that second Logstash run will have the following where statement:
where timestamp > 2019-04-04 09:40:02.
Does this mean that DB entry 2 will be omitted entirely by Logstash? Is that possible? Is there a time offset or a buffer that makes Logstash 'wait' few seconds to make sure that no such situation will happen?