# Issue of protobuf for logstash codec

**URL:** <https://discuss.elastic.co/t/issue-of-protobuf-for-logstash-codec/100102>\
**Category:** Logstash\
**Created:** [September 11, 2017, 4:58pm UTC](https://discuss.elastic.co/t/issue-of-protobuf-for-logstash-codec/100102 "2017-09-11T16:58:47Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![fluency03](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fluency03/32/21977_2.png) [@fluency03](https://discuss.elastic.co/u/fluency03)\
**Post date:** [September 11, 2017, 4:58pm UTC](https://discuss.elastic.co/t/issue-of-protobuf-for-logstash-codec/100102/1 "2017-09-11T16:58:47Z")

</div>

I am trying to use protobuf for logstash codec. I am following :

- [https://github.com/codekitchen/ruby-protocol-buffers](https://github.com/codekitchen/ruby-protocol-buffers)
- [https://github.com/logstash-plugins/logstash-codec-protobuf](https://github.com/logstash-plugins/logstash-codec-protobuf)
- [http://tech.trivago.com/2016/01/19/logstash\_protobuf\_codec/](http://tech.trivago.com/2016/01/19/logstash_protobuf_codec/)

However, I cannot start my logstash.

```
bin/logstash -f logstash-test-1.conf --config.reload.automatic --debug

```

Here is the error:

```
...
[2017-09-11T18:46:44,887][DEBUG][logstash.inputs.twitter] config LogStash::Inputs::Twitter/@rate_limit_reset_in = 300
[2017-09-11T18:46:44,970][DEBUG][logstash.plugins.registry] On demand adding plugin to the registry {:name=>"protobuf", :type=>"codec", :class=>LogStash::Codecs::Protobuf}
[2017-09-11T18:46:44,974][DEBUG][logstash.codecs.protobuf] config LogStash::Codecs::Protobuf/@class_name = "Twitter"
[2017-09-11T18:46:44,974][DEBUG][logstash.codecs.protobuf] config LogStash::Codecs::Protobuf/@include_path = ["/Users/fluency03/Workplace/protobuf/TwitterSimple.pb.rb"]
[2017-09-11T18:46:44,974][DEBUG][logstash.codecs.protobuf] config LogStash::Codecs::Protobuf/@id = "16ab258d8e30f65d80c0bb85353e47740f7c9971-2"
[2017-09-11T18:46:44,975][DEBUG][logstash.codecs.protobuf] config LogStash::Codecs::Protobuf/@enable_metric = true
[2017-09-11T18:46:44,975][DEBUG][logstash.codecs.protobuf] Including protobuf file: /Users/fluency03/Workplace/protobuf/TwitterSimple.pb.rb
[2017-09-11T18:46:44,982][ERROR][logstash.agent] Cannot create pipeline {
:reason=>"Twitter is not a class", 
:backtrace=>[
"/Users/fluency03/Workplace/protobuf/TwitterSimple.pb.rb:9:in `(root)'", 
"org/jruby/RubyKernel.java:1040:in `require'", 
"/Users/fluency03/Workplace/elastic/logstash-5.5.2-1/vendor/bundle/jruby/1.9/gems/polyglot-0.3.5/lib/polyglot.rb:65:in `require'", 
"/Users/fluency03/Workplace/elastic/logstash-5.5.2-1/vendor/local_gems/797f65a3/logstash-codec-protobuf-1.0.3/lib/logstash/codecs/protobuf.rb:1:in `(root)'", 
"/Users/fluency03/Workplace/elastic/logstash-5.5.2-1/vendor/local_gems/797f65a3/logstash-codec-protobuf-1.0.3/lib/logstash/codecs/protobuf.rb:163:in `require_with_metadata_analysis'", 
"/Users/fluency03/Workplace/elastic/logstash-5.5.2-1/vendor/local_gems/797f65a3/logstash-codec-protobuf-1.0.3/lib/logstash/codecs/protobuf.rb:207:in `require_pb_path'", 
"org/jruby/RubyArray.java:1613:in `each'", 
"/Users/fluency03/Workplace/elastic/logstash-5.5.2-1/vendor/local_gems/797f65a3/logstash-codec-protobuf-1.0.3/lib/logstash/codecs/protobuf.rb:71:in `register'", 
"/Users/fluency03/Workplace/elastic/logstash-5.5.2-1/vendor/local_gems/797f65a3/logstash-codec-protobuf-1.0.3/lib/logstash/codecs/protobuf.rb:71:in `register'", 
"/Users/fluency03/Workplace/elastic/logstash-5.5.2-1/logstash-core/lib/logstash/codecs/base.rb:20:in `initialize'", 
"/Users/fluency03/Workplace/elastic/logstash-5.5.2-1/logstash-core/lib/logstash/pipeline.rb:109:in `plugin'", 
"org/jruby/RubyKernel.java:1079:in `eval'", 
"(eval):12:in `initialize'", "/Users/fluency03/Workplace/elastic/logstash-5.5.2-1/logstash-core/lib/logstash/pipeline.rb:72:in `initialize'", 
"/Users/fluency03/Workplace/elastic/logstash-5.5.2-1/logstash-core/lib/logstash/pipeline.rb:156:in `initialize'", 
"/Users/fluency03/Workplace/elastic/logstash-5.5.2-1/logstash-core/lib/logstash/agent.rb:286:in `create_pipeline'", 
"/Users/fluency03/Workplace/elastic/logstash-5.5.2-1/logstash-core/lib/logstash/agent.rb:95:in `register_pipeline'", 
"/Users/fluency03/Workplace/elastic/logstash-5.5.2-1/logstash-core/lib/logstash/runner.rb:314:in `execute'", 
"/Users/fluency03/Workplace/elastic/logstash-5.5.2-1/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:67:in `run'", 
"/Users/fluency03/Workplace/elastic/logstash-5.5.2-1/logstash-core/lib/logstash/runner.rb:209:in `run'", 
"/Users/fluency03/Workplace/elastic/logstash-5.5.2-1/lib/bootstrap/environment.rb:71:in `(root)'"]}
[2017-09-11T18:46:44,988][DEBUG][logstash.agent] starting agent
[2017-09-11T18:46:44,991][DEBUG][logstash.agent] Starting puma
[2017-09-11T18:46:44,991][DEBUG][logstash.instrument.periodicpoller.os] PeriodicPoller: Stopping
[2017-09-11T18:46:44,991][DEBUG][logstash.agent] Trying to start WebServer {:port=>9600}
[2017-09-11T18:46:44,993][DEBUG][logstash.instrument.periodicpoller.jvm] PeriodicPoller: Stopping
[2017-09-11T18:46:44,994][DEBUG][logstash.instrument.periodicpoller.persistentqueue] PeriodicPoller: Stopping
[2017-09-11T18:46:44,994][DEBUG][logstash.api.service] [api-service] start

```

Why I got the error: `Twitter is not a class`?

Here is my logstash config:

```
...
output {
  kafka {
    bootstrap_servers => "localhost:9092"
    codec => protobuf {
      class_name => "Twitter"
      include_path => ['/Users/fluency03/Workplace/protobuf/TwitterSimple.pb.rb']
    }
    topic_id => "blockchain-tweets"
  }
# stdout {
# codec => rubydebug
# }
}

```

This is only giving error when I use Kafka as output of logstash. When I use Kafka as input and add codec of protobuf, it can successfully load the `pb.rb` class.

* * *

Here is my `TwitterSimple.proto`:

```
syntax = "proto3";

message Twitter {
  string message = 1;
}

```

Here is my generated `TwitterSimple.pb.rb` after `ruby-protoc TwitterSimple.proto`:

```
#!/usr/bin/env ruby
# Generated by the protocol buffer compiler. DO NOT EDIT!

require 'protocol_buffers'

# forward declarations
class Twitter < ::ProtocolBuffers::Message; end

class Twitter < ::ProtocolBuffers::Message
  set_fully_qualified_name "Twitter"

  optional :string, :message, 1
end
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 9, 2017, 4:58pm UTC](https://discuss.elastic.co/t/issue-of-protobuf-for-logstash-codec/100102/2 "2017-10-09T16:58:58Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.