Documentation on in_event and expect in Ruby filter test framework?

Elastic Stack Logstash
1

I started working on a Ruby script to be called from my Logstash ruby filter.

The official documentation mentions a test framework here

There is an example test provided

test "drop percentage 100%" do
  parameters do
    { "percentage" => 1 }
  end

  in_event { { "message" => "hello" } }

  expect("drops the event") do |events|
    events.size == 0
  end
end

Is there any documentation for in_event? Do I just paste in a full JSON document to test on?

Does expect work like the expect in RSpec? If not, is there any documentation on how to use it?

2

Sorry, too late for me to edit the OP with my additional questions about the example test provided in the official docs:

  1. Where does the parameters variable come from?

  2. Can we type anything we want into the expect argument? For example is the above test still going to run as expected if we had typed expect("blow away event")?

  3. How do I get the output of the filter function in the test? The example test just looks at the number of events. The official docs say filter returns a list of events. It would have been nice to see the expect portion of the test take a look at - at the very least - one value inside one of the events in the list of events that filter is supposed to return.

3

Here is my test, based on guessing what might work since I cannot find the proper documentation:

test "get task array" do
  in_event {
    {
      "name":"job1",
      "error":[
         {
         "task":"75fc",
         "message":{
            "ietf-restconf:errors":{
               "error":[
                  {
                     "error-type":"application",
                     "error-tag":"malformed-message",
                     "error-path":"/pathto/problem",
                     "error-message":"missing element: name in thepath"
                  }
               ]
            }
         },
         "timestamp":1.695060733555E+12
        },
       {
         "task":"job",
         "message":"Job has no available transitions. 6649, cb04, f5dd could have led to the workflow end task, but did not. These tasks performed in a way that the end of the workflow could not be reached.",
         "timestamp":1.69506073357E+12
        }
      ]
    }
  }

  expect("extracts the task array") do |events|
    puts events
    events.size == 2
  end
end

I am getting a test failure. I tried to use the puts statement to see what is actually inside events in the expect block, but the puts is not producing any output. What do I need to do?

4

I had never tried testing ruby scripts before this, but having poked around in the source...

I think in_event should be a hash or an array of hashes, each of which is used to create an event (@timestamp and @version get added).

I think the do |events| is passed the array of events return by your filter function. So your test should be

events[0].get("tasks").size == 2

If I add -t to my logstash command line I then get

[2023-10-06T19:21:55,283][INFO ][logstash.filters.ruby.script] Test run complete {:script_path=>"/home/user/.../script.rb", :results=>{:passed=>1, :failed=>0, :errored=>0}}

Any puts calls in either the filter function or the expect bit write to stdout as expected.

However puts events just produces

[#<LogStash::Event:0x29eeb1bb>]

and puts events[0] produces

2023-10-06T23:32:54.237298312Z %{host} %{message}

because it calls the .to_s or inspect of the event and that method is not useful. You can get the actual event using puts events[0].to_hash.

1 Like
5

It seems like the official documentation mentions the use of a test framework with an example test case provided. However, it's unclear if there's documentation available for the "in_event" function and how to use it effectively. Similarly, more information about how "expect" works, especially in comparison to RSpec, would be helpful for users trying to understand its usage. AC Football Cases

6

Thanks! I went back and forth on whether or not to put the ruby filter inline, or put it in a separate .rb file. I chose the latter to use the testing framework, because there are so many possible points of failure in the pipeline config file already. Testing the ruby filter by itself looks like a good idea.

The explanation of in_event makes sense

I modified the expect mehod as follows

expect("extracts the task array") do |events|
    puts "EVENTS ARRAY:"
    puts events[0].to_hash
    events[0].get(["error_task_array"]).count == 2
  end # expect

I can see in the output that error_task_array has 2 elements as expected

EVENTS ARRAY:
{"error_task_array"=>["75fc", "job"], "@timestamp"=>2023-10-09T19:06:52.475893048Z, "error"=>[{"timestamp"=>1695060733555.0, "task"=>"75fc", "message"=>{"ietf-restconf:errors"=>{"error"=>[{"error-message"=>"missing element: name in thepath", "error-type"=>"application", "error-path"=>"/pathto/problem", "error-tag"=>"malformed-message"}]}}}, {"timestamp"=>1695060733570.0, "task"=>"job", "message"=>"Job has no available transitions. 6649, cb04, f5dd could have led to the workflow end task, but did not. These tasks performed in a way that the end of the workflow could not be reached."}], "name"=>"job1", "@version"=>"1"}

However the test continues to fail

[ERROR] 2023-10-09 19:15:13.373 [LogStash::Runner] expectcontext - ***TEST RAISED ERROR: 'get task array extracts the task array'*** {"exception"=>"#<TypeError: no implicit conversion of Array into String>"

I cannot figure out the proper way to get the number of elements in error_task_array. Besides count I also tried size and length.

7

I got the test to pass by cheating in this manner:

expect("extracts the task array") do |events|
    events.size != 0
    #taskArray = events[0].get(["error_task_array"])
    #taskArray != null
end # expect

It's a shame that I cannot implement a proper test for now but I need to get a move on.

I opened an issue on Github against the ruby filter. I see there are a bunch of open issues already regarding the lack of documentation. However the issue I opened is specifically about not being able to check the size of an array.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.