Ruby filter to get data for performance alerting

Elastic Stack Logstash
1

Hi !
I'm working with json below to configure trigger when "free" will be near to 0,

{
           "@timestamp" => 2021-10-13T08:51:00.395Z,
               "type" => "metric",
           "@version" => "1",
               "user" => "performance",
          "timeStamp" => "2021-10-13T10:51:00.390781+02:00",
            "version" => 2,
                 "id" => "e203a02f-2acf-4deb-8230-bdd9057afe20",
               "tags" => [
        [0] "applogs"
    ],
            "warning" => "false",
            "payload" => {
                 "version" => 1,
           "collectorName" => "peek-filesystem",
        "collectorVersion" => "1.5.23+803659e",
            "measurements" => [
            [ 0] {
                "resourceType" => "system_filesystem",
                  "properties" => {
                           "options" => "rw,seclabel,nosuid,size=32883064k,nr_inodes=8220766,mode=755",
                              "path" => "/opt/was",
                      "resourceName" => "win12345.ec.domain|ext2|/opt/was",
                    "filesystemType" => "ext2",
                             "mount" => "ext2"
                },
                     "metrics" => [
                    [0] {
                        "detailLevel" => 1,
                              "value" => 32112,
                               "type" => "gauge",
                               "unit" => "MB",
                               "name" => "free"
                    },
                    [1] {
                        "detailLevel" => 1,
                              "value" => 32112,
                               "type" => "gauge",
                               "unit" => "MB",
                               "name" => "available"
                    },
                    [2] {   [...]

There's array of measurements and another array of properties for each filesystem.
Could You help me with ruby filter to get that value of free and compare it to let's say 10MB for each of the filesystems.

Thanks in advance,

2

That is not a well specified problem. I am guessing your event includes an array of hashes ([payload][measurements]), each of which has an array of metrics and a hash of properties (not an array). If this guess is wrong you will need to edit the code appropriately.

You have also given no indication of what you want to do with the result of the comparison, so I provide multiple options...

    ruby {
        code => '
            m = event.get("[payload][measurements]")
            if m.is_a? Array
                m.each_index { |x|
                    metrics = m[x]["metrics"]
                    dodgyDisks = []
                    metrics.each { |y|
                        if y["name"] == "free"
                            if y["unit"] != "MB"
                                # You should grab some code from the bytes filter on github and
                                # re-write this entire block with sizes normalized to bytes
                                # https://github.com/logstash-plugins/logstash-filter-bytes/blob/master/lib/logstash/filters/bytes.rb
                            else
                                # What to do? What to do?
                                if y["value"] < 10
                                    m[x]["properties"]["freeSpaceWarning"] = true
                                else
                                    m[x]["properties"]["freeSpaceWarning"] = false
                                end
                                # Or maybe
                                if y["value"] < 10
                                    event.set("diskSpaceIssue", true)
                                end
                                # Or even
                                if y["value"] < 10
                                    dodgyDisks << m[x]["properties"]["path"]
                                end
                            end
                        end
                        if dodgyDisks != []
                            event.set("dodgyDisks", dodgyDisks)
                        end
                    }
                }
                event.set("[payload][measurements]", m)
            end
        '
    }

Note that this assumes that every resource is a "system_filesystem". It is an outline of an approach, not production ready code.

1 Like
3

Thanks a lot ! That was very helpful and pushed me into right direction ! I have last question, if I want to get rid of those fields from event, leaving just "diskSpaceIssue" should I use mutate on it or rather prune filter to add just allowed parameters ?

4

If you do not know the names of all the fields you want to remove then prune would be better.

5

Do You know how to make value of dodgyDisks to contain all the paths ? Right now there's only the last value of path which has free space issue e.g /var/log is the [14]th element with disk space issue mentioned in dodgyDisks but there were also 2 other on [4]th place and [10]th place.

Thanks !

6

I would expect it to contain all of the paths that match the condition. It is array and the paths are appended to it using <<.

7

Thank You for the reply, for some reason << doesn't increase dodgyDisks. You've helped me a lot, I'll work on figuring it out.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.