# Unbale to decode Message using protobuf logstahs plusgin which is encoded using python or ruby?

**URL:** <https://discuss.elastic.co/t/unbale-to-decode-message-using-protobuf-logstahs-plusgin-which-is-encoded-using-python-or-ruby/132414>\
**Category:** Logstash\
**Created:** [May 18, 2018, 4:28am UTC](https://discuss.elastic.co/t/unbale-to-decode-message-using-protobuf-logstahs-plusgin-which-is-encoded-using-python-or-ruby/132414 "2018-05-18T04:28:24Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![RameshNagargoje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rameshnagargoje/32/30212_2.png) [@RameshNagargoje](https://discuss.elastic.co/u/RameshNagargoje)\
**Post date:** [May 18, 2018, 4:28am UTC](https://discuss.elastic.co/t/unbale-to-decode-message-using-protobuf-logstahs-plusgin-which-is-encoded-using-python-or-ruby/132414/1 "2018-05-18T04:28:24Z")

</div>

Here is message.proto

syntax = "proto3";  
message SearchRequest {  
string query = 1;  
int32 page\_number = 2;  
int32 result\_per\_page = 3;  
}

compiled using sudo protoc --python\_out=. message.proto

which will gives message\_pb.py as

# Generated by the protocol buffer compiler. DO NOT EDIT!

# source: message.proto

import sys  
\_b=sys.version\_info[0]\<3 and (lambda x:x) or (lambda x:x.encode('latin1'))  
from google.protobuf import descriptor as \_descriptor  
from google.protobuf import message as \_message  
from google.protobuf import reflection as \_reflection  
from google.protobuf import symbol\_database as \_symbol\_database  
from google.protobuf import descriptor\_pb2

# @@protoc\_insertion\_point(imports)

\_sym\_db = \_symbol\_database.Default()

DESCRIPTOR = \_descriptor.FileDescriptor(  
name='message.proto',  
package='',  
syntax='proto3',  
serialized\_pb=\_b('\n\rmessage.proto"L\n\rSearchRequest\x12\r\n\x05query\x18\x01 \x01(\t\x12\x13\n\x0bpage\_number\x18\x02 \x01(\x05\x12\x17\n\x0fresult\_per\_page\x18\x03 \x01(\x05\x62\x06proto3')  
)  
\_sym\_db.RegisterFileDescriptor(DESCRIPTOR)

\_SEARCHREQUEST = \_descriptor.Descriptor(  
name='SearchRequest',  
full\_name='SearchRequest',  
filename=None,  
file=DESCRIPTOR,  
containing\_type=None,  
fields=[  
\_descriptor.FieldDescriptor(  
name='query', full\_name='SearchRequest.query', index=0,  
number=1, type=9, cpp\_type=9, label=1,  
has\_default\_value=False, default\_value=\_b("").decode('utf-8'),  
message\_type=None, enum\_type=None, containing\_type=None,  
is\_extension=False, extension\_scope=None,  
options=None),  
\_descriptor.FieldDescriptor(  
name='page\_number', full\_name='SearchRequest.page\_number', index=1,  
number=2, type=5, cpp\_type=1, label=1,  
has\_default\_value=False, default\_value=0,  
message\_type=None, enum\_type=None, containing\_type=None,  
is\_extension=False, extension\_scope=None,  
options=None),  
\_descriptor.FieldDescriptor(  
name='result\_per\_page', full\_name='SearchRequest.result\_per\_page', index=2,  
number=3, type=5, cpp\_type=1, label=1,  
has\_default\_value=False, default\_value=0,  
message\_type=None, enum\_type=None, containing\_type=None,  
is\_extension=False, extension\_scope=None,  
options=None),  
],  
extensions=[  
],  
nested\_types=[],  
enum\_types=[  
],  
options=None,  
is\_extendable=False,  
syntax='proto3',  
extension\_ranges=[],  
oneofs=[  
],  
serialized\_start=17,  
serialized\_end=93,  
)

DESCRIPTOR.message\_types\_by\_name['SearchRequest'] = \_SEARCHREQUEST

SearchRequest = \_reflection.GeneratedProtocolMessageType('SearchRequest', (\_message.Message,), dict(  
DESCRIPTOR = \_SEARCHREQUEST,  
**module** = 'message\_pb2'

# @@protoc\_insertion\_point(class\_scope:SearchRequest)

))  
\_sym\_db.RegisterMessage(SearchRequest)

# @@protoc\_insertion\_point(module\_scope)

and one simple python script is used to encode message as test\_protobug.py

import message\_pb2  
import sys

message = message\_pb2.SearchRequest()  
message.query = "First message query"  
message.page\_number = 123  
message.result\_per\_page = 10  
f = open("firstmessage.log", "w")

print "Encoded: %s "%message.SerializeToString()  
f.write(message.SerializeToString())  
f.close()

f = open("firstmessage.log", "r")  
message.ParseFromString(f.read())  
f.close()  
print message

for logstash message.proto is compiled and converted in message into ruby as message\_pd.rb as

#!/usr/bin/env ruby

# Generated by the protocol buffer compiler. DO NOT EDIT!

require 'protocol\_buffers'

# forward declarations

class SearchRequest \< ::ProtocolBuffers::Message; end

class SearchRequest \< ::ProtocolBuffers::Message  
set\_fully\_qualified\_name "SearchRequest"

required :string, :query, 1  
required :int32, :page\_number, 2  
required :int32, :result\_per\_page, 3  
end

and logstash conf file is as

input  
{  
file  
{  
path =\> "/home/qabuilder/protobuf/firstmessage.log"  
start\_position =\> "beginning"

```
	codec => protobuf
	{
        class_name => "SearchRequest"
        include_path => ["/tmp/message_pb.rb"]
        protobuf_version => 3
	}
	 
}

```

}

output  
{  
stdout  
{  
codec =\> json  
}  
}

gives empty fields as

[2018-05-17T08:18:13,902][DEBUG][logstash.runner] node.name: "rn-dt1"  
[2018-05-17T08:18:17,785][DEBUG][logstash.pipeline] filter received {"event"=\>{"result\_per\_page"=\>0, "page\_number"=\>0, "query"=\>"", "@timestamp"=\>2018-05-17T15:18:17.662Z, "path"=\>"/home/qabuilder/protobuf/firstmessage.log", "host"=\>"rn-dt1", "@version"=\>"1"}}  
[2018-05-17T08:18:17,792][DEBUG][logstash.pipeline] output received {"event"=\>{"result\_per\_page"=\>0, "page\_number"=\>0, "query"=\>"", "@timestamp"=\>2018-05-17T15:18:17.662Z, "path"=\>"/home/qabuilder/protobuf/firstmessage.log", "host"=\>"rn-dt1", "@version"=\>"1"}}  
{"result\_per\_page":0,"page\_number":0,"query":"","@timestamp":"2018-05-17T15:18:17.662Z","path":"/home/qabuilder/protobuf/firstmessage.log","host":"rn-dt1","@version":"1"}[2018-05-17T08:18:19,175][DEBUG][logstash.instrument.periodicpoller.cgroup] Error, cannot retrieve cgroups information {:exception=\>"Errno::ENOENT", :message=\>"No such file or directory - /sys/fs/cgroup/cpuacct/cpu.cfs\_period\_us"}

Expected o/p is

query = "First message query"  
page\_number = 123  
result\_per\_page = 10

but it gives as

{  
"result\_per\_page": 0,  
"page\_number": 0,  
"query": ""  
}

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 15, 2018, 4:28am UTC](https://discuss.elastic.co/t/unbale-to-decode-message-using-protobuf-logstahs-plusgin-which-is-encoded-using-python-or-ruby/132414/2 "2018-06-15T04:28:25Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.