Ruby filter not running all the code on every event?!

[Miguel_Oliveira]

Hello everyone,

I'm implementing a use case where I'm reading events from a Kafka topic and then I use a Ruby filter to run some custom code (version 6.6.2).
The idea is to find some date fields that can have multiple date formats and create additional fields with a standard date format.
However, when running the Ruby code below, the behavior we're seeing is that some events get all the new fields, some events only have some of the new fields and some events don't get any of the new fields. And note that in all cases all of them have one of the date formats being tested.

  ruby {
    code => "
      event.set('opco_id', event.get('[@metadata][kafka][topic]')[18..19].downcase)

      # New Dates
      dates_to_parse = ['[Custom][A]', 'TimeStamp', '[Custom][B]', '[Custom][C]', '[Custom][D]', '[Custom][E]', '[Custom][F]']
      dates_new_field = ['dt_A','dt_EventTimestamp','dt_B','dt_C','dt_D','dt_E','dt_F']

      dates_to_parse.each_with_index { |date, i|
        date_formats = ['%c', '%FT%TZ', '%FT%T.%LZ', '%F %T.%L T', '%Y%m%d-%H%M%S', '%F %l:%M:%S.%L %P T']
        parsed_date = nil
        date_formats.each { |f|
           parsed_date ||= DateTime.strptime(event.get(date), f) rescue nil
           break if !(parsed_date.nil?)
        }
        if !(parsed_date.nil?)
        then
          event.set(dates_new_field[i],parsed_date.strftime('%FT%T.%LZ'))
        end
      }
    "
  }

Does anyone have an idea on what might be happening?
Or how to further debug/identify the root cause?

Thank you.
Best regards,
Miguel