S3 input dont delete/backup files after processing

froheik · March 10, 2022, 3:33am

Found some interesting code

logstash-plugins/logstash-input-s3/blob/main/lib/logstash/inputs/s3.rb#L381

      
        
              else
                return false
              end
            end
            
            
def process_log(queue, log)
              @logger.debug("Processing", :bucket => @bucket, :key => log.key)
              object = @s3bucket.object(log.key)
            
            
  filename = File.join(temporary_directory, File.basename(log.key))
              if download_remote_file(object, filename)
                if process_local_log(queue, filename, object)
                  if object.last_modified == log.last_modified
                    backup_to_bucket(object)
                    backup_to_dir(filename)
                    delete_file_from_bucket(object)
                    FileUtils.remove_entry_secure(filename, true)
                    sincedb.write(log.last_modified)
                  else
                    @logger.info("#{log.key} is updated at #{object.last_modified} and will process in the next cycle")
                  end

As i understand - plugin compare last_modified of object and log, and according to my log - postpone object processing to next cycle.
But what is it mean? In next cycle it repeating.
Why required compare time of last modifying?
How to get that two timestamps is equal? May be it can skipped?