Time difference between two fields in a csv using ruby plugin

Elastic Stack Logstash
1

Hi all

New to the forum ,ive been trying to get time difference between two fields in a csv using ruby plugin but i am new to logstash and using ruby can somebody please assist with the syntax of the ruby filter
(running logstash 6.1.1)

My .conf file filter is as follows:

filter {
if [projectid] == "Sample" {
csv {
     columns => ["DOMAIN","NAME","ORDERNUM","CATEGORY","SUBCATEGORY","CURRENT_PRIORITY","STATE","CREATED","RESOLVED","CLOSED","VALUE","COMPLETED_HOUR"]
     convert => {
         "COMPLETED_HOUR" => "integer"
     }
 }

Drop Header Rows

if "DOMAIN" in [DOMAIN] {
   drop {}
}

Date format

date {
  match => [ "CREATED", "dd/MM/yyyy HH:mm", "ISO8601"]
 target => "CREATED"
}
date {
  match => [ "RESOLVED", "dd/MM/yyyy HH:mm", "ISO8601"]
target => "RESOLVED"
}
date {
  match => [ "CLOSED", "dd/MM/yyyy HH:mm", "ISO8601"]
target => "CLOSED"
}

Ruby filter

ruby {
    init => "require 'time'"
    code => "
      diff = event['CREATED'] - event['RESOLVED']
      event['timediff'] = diff;
      "
    add_tag => [ "calculated_time_difference" ]
  }
}
}

I tried the ruby filter above but i am getting the error (see below) :

[ERROR] 2018-03-01 15:46:57.531 [Ruby-0-Thread-11@[main]>worker2: /usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:392] ruby - Ruby exception occurred: Direct event field references (i.e. event['field']) have been disabled in favor of using event get and set methods (e.g. event.get('field')). Please consult the Logstash 5.0 breaking changes documentation for more details.

Any help will be much appreciated ,TIA

2

As the message says, you should be using event.get to access fields, and event.set to add them. For example, this code calculates the differences (in seconds) between two times. It does something slightly different to what you want, and comes from an aggregate filter, so is structured differently, but accesses and sets fields, so should give you some idea of what to do.

  code => "
    require 'time';
    starttime = Time.iso8601(map['starttime'].to_s).to_f;
    endtime   = Time.iso8601(event.get('@timestamp').to_s).to_f;
    event.set('overall_timetaken', endtime - starttime);
  "
4

Hi Badger , The below seemed to work for me

ruby {
    init => "require 'time'"
    code => "
           starttime = Time.iso8601(event.get('CREATED').to_s).to_f;
           endtime   = Time.iso8601(event.get('RESOLVED').to_s).to_f;
           event.set('time_diff', endtime - starttime);
            "
    add_tag => [ "calculated_time_difference" ]
  }

Thank you for your input :slight_smile:

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.