How to process/filter output from SNMP input plugin for logstash

Elastic Stack Logstash
1

hi, I am retrieving SNMP data from our webcast servers. I would like to know how to process the results in a filter stage in logstash e.g

    {
          "httpClientCountGetUserQueryStr.1.1.1.1.3" => "liveoutside/smil:1234_camera_all.smil/playlist.m3u8",
                                          "@version" => "1",
        "httpClientCountGetUserQueryStr.1.1.1.1.144" => "liveoutside/1234_camera_720p/playlist.m3u8",
                                            "fields" => {
            "document_type" => "wowzasnmp"
        },
          "httpClientCountGetIsFirstChunk.1.1.1.1.3" => "118.185.62.42",
                                        "@timestamp" => 2020-07-01T22:28:48.704Z,
        "httpClientCountGetIsFirstChunk.1.1.1.1.144" => "118.141.203.207"
    }

I could have a big number of httpClientCountGetIsFirstChunk & httpClientCountGetUserQueryStr not always in a 1 to 1 relationship as a client could demand several streams from the server.

Just for completeness this is how I collect data:

    input {
            snmp {
                     walk => [".1.3.6.1.4.1.46706.100.70.1.1.1.5",".1.3.6.1.4.1.46706.100.70.1.1.1.28"]
                     hosts => [{host => "udp:wowzaqa.domain.com/1611" community => "public" version => "2c"  retries => 2  timeout => 1000}]
                     mib_paths => ["/usr/share/snmp/mibs/WOWZASTREAMINGENGINE.dic"]
                     oid_path_length => 6
                     add_field => {"[fields][document_type]" => "wowzasnmp"}
            }
    }

Thank you

2

That really depends on what output you want.

3

Indeed I didnt explain. What I need to do is to parse the streams and get for example: for liveoutside/1234_camera_720p/playlist.m3u8 two fields appname: liveoutside and streamname:1234_camera. Then I need to group the different IP's that are connected to each pair streamname and appname. In the example it would be 2 IP for just one pair. But I could have a variety of results e.g.

{
                                      "@version" => "1",
    "httpClientCountGetIsFirstChunk.1.1.1.1.158" => "188.189.117.200",
    "httpClientCountGetUserQueryStr.1.1.1.1.155" => "liveoutside/smil:1234_camera_all.smil/playlist.m3u8",
    "httpClientCountGetUserQueryStr.1.1.1.1.158" => "liveoutside/smil:1234_camera_all.smil/playlist.m3u8",
                                        "fields" => {
        "document_type" => "wowzasnmp"
    },
    "httpClientCountGetIsFirstChunk.1.1.1.1.159" => "188.189.117.200",
      "httpClientCountGetIsFirstChunk.1.1.1.1.3" => "188.189.64.42",
    "httpClientCountGetIsFirstChunk.1.1.1.1.155" => "188.189.117.200",
                                    "@timestamp" => 2020-07-01T22:31:48.773Z,
    "httpClientCountGetIsFirstChunk.1.1.1.1.157" => "188.189.117.200",
      "httpClientCountGetUserQueryStr.1.1.1.1.3" => "liveoutside/smil:1234_camera_all.smil/playlist.m3u8",
    "httpClientCountGetIsFirstChunk.1.1.1.1.156" => "188.189.117.200",
    "httpClientCountGetUserQueryStr.1.1.1.1.156" => "liveoutside/smil:1234_camera_all.smil/playlist.m3u8",
    "httpClientCountGetUserQueryStr.1.1.1.1.144" => "liveoutside/1234_camera_720p/playlist.m3u8",
    "httpClientCountGetUserQueryStr.1.1.1.1.159" => "liveoutside/smil:1234_camera_all.smil/playlist.m3u8",
    "httpClientCountGetUserQueryStr.1.1.1.1.157" => "liveoutside/smil:1234_camera_all.smil/playlist.m3u8",
    "httpClientCountGetIsFirstChunk.1.1.1.1.144" => "128.171.209.207"
}

Thank you