Logstash unable to decode Cisco protobuf

[Scottsdash (Scott)]

Hoping someone has configured logstash to receive Cisco telemetry before, I'm able to receive the events but can't decode the telemetry.

I'm getting the folowing error from logstash:

[WARN ] 2020-09-09 05:18:01.074 [nioEventLoopGroup-2-1] protobuf - Couldn't decode protobuf: #<RuntimeError: Protocol message contained an invalid tag (zero).>.
{
          "host" => "10.0.0.130",
          "tags" => [
        [0] "_protobufdecodefailure"
    ],
          "port" => 60779,
       "message" => "\x00\x00\x9A\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x95\bH\x12\x90\x01\n\x06Router\x1A\x01127Cisco-IOS-XE-process-cpu-oper:cpu-usage/cpu-utilizationP\xEE\xE8\xB1\x8A\xC7.ZC\b\xEE\xE8\xB1\x8A\xC7.z\x06\x12\x04keysz2\x12\acontentz\x10\x12\ffive-seconds8\x00z\x15\x12\x13cpu-usage-processes",
    "@timestamp" => 2020-09-09T05:18:01.072Z,
      "@version" => "1"

Rotuer : Cisco CSRv 16.12 (IOS XR)
Protofile is sourced from Cisco: https://github.com/cisco/bigmuddy-network-telemetry-proto
Specifically I'm using the telemetry.proto file which has been compiled with the google protoc binaries.

Router configuration:

telemetry ietf subscription 1
 encoding encode-kvgpb
 filter xpath /process-cpu-ios-xe-oper:cpu-usage/cpu-utilization/five-seconds
 stream yang-push
 update-policy periodic 500
 receiver ip address 10.0.0.80 5000 protocol grpc-tcp

Logstash configuration file:

input {
  tcp {
    port => 5000
    codec => protobuf {
      class_name => "telemetry.Telemetry"
      class_file => '/home/scott/proto/telemetry_pb.rb'
      protobuf_root_directory => "/home/scott/proto/"
      protobuf_version => 3
    }
  }
}
output {
  elasticsearch {
    hosts => ["localhost:9200"]
  }
  stdout {
    codec => rubydebug
  }
}

[Sako]

Hi!

I try to achieve the same and run into the same error. I try to get interface statistics.
For me it look like there are two messages in the protobuf, the first is the interface name and the second are the actual interface counters.

In the LS plugin I can only specify one of the two messages / classes...

[Scottsdash (Scott)]

Thanks for the response.

I was starting to think perhaps Cisco had a non standard implementation of gRPC how I've been able to successfully index data with the TICK stack. Note I'm using key value gRPC and with TICK i didn't require the proto files.
I had hoped to do a comparison of the two, with a preference for elastic for both logging and metric data.

Wondering if anyone has had any luck receiving Cisco telmetry? Cisco have a bunch of out dated plugins that no longer seem to work, they don't seem to be being maintained so I'm reluctant to use them even if I could compile / install them.

[Sako]

I think this is related to https://github.com/logstash-plugins/logstash-codec-protobuf/issues/23

I tried running one of the old plugins but they are compatible up to LS 3.0, so they did not work with 7.9.1 any more...

right now the only way to do this is with the self describing option and then you get the key/value format. but this is more overhead on both ends, the Cisco device and LS and does not scale that much for metrics... I need to design for about 100 devices with 50-300 interfaces each sending metrics data every 5 seconds. 30000 interfaces sending every 5 seconds is a lot of metrics and I doubt that the key/value format will scale.