# Logstash - input filter ruby. Persisted vs memory queue: different way to store a blob column from Oracle

**URL:** <https://discuss.elastic.co/t/logstash-input-filter-ruby-persisted-vs-memory-queue-different-way-to-store-a-blob-column-from-oracle/209481>\
**Category:** Logstash\
**Created:** [November 26, 2019, 11:02am UTC](https://discuss.elastic.co/t/logstash-input-filter-ruby-persisted-vs-memory-queue-different-way-to-store-a-blob-column-from-oracle/209481 "2019-11-26T11:02:16Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![evecitas89](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/evecitas89/32/56761_2.png) [@evecitas89](https://discuss.elastic.co/u/evecitas89)\
**Post date:** [November 26, 2019, 11:02am UTC](https://discuss.elastic.co/t/logstash-input-filter-ruby-persisted-vs-memory-queue-different-way-to-store-a-blob-column-from-oracle/209481/1 "2019-11-26T11:02:16Z")

</div>

Hello everyone.

I have a Logstash pipeline reading from Oracle database a Blob column, named INPUT\_MESSAGE.  
In the pipepiline.yml I was using this configuration for the pipeline:

**queue.type: memory**

In the pipeline.config file, my input ruby filter correctly manages the Blob input in this way:

> ruby {  
> init =\>"require 'zlib'; require 'stringio';"  
> code =\>"  
> message = event.get('input\_message')  
> unless message.nil?  
> if (message.bytes[0] == 1 || message.bytes[0] == 4)  
> messageContentReader = Zlib::GzipReader.new(StringIO.new(message[1...message.length]))  
> messageContent = messageContentReader.read  
> messageContentReader.close  
> else  
> messageContent = message[1...message.length].to\_s  
> end  
> event.set('input\_message', messageContent)  
> event.set('input\_message\_size', messageContent.bytesize)  
> end  
> "  
> }

I've put a Logger Info to track value of the variable "message". It is, for example:

#\<Sequel::SQL::Blob:0x7e0 bytes=89 start="\x01\x1F\x8B\b\x00\x00\x00\x00\x00\x04" end="\x01\x004\x88@\xA6Y\x00\x00\x00"\>

However, due to project requirements, I had to switch to a persisted queue configuration:

**queue.type: persisted**

In this case, the way of Logstash of storing the column value is different: the value of the variable message comes up to be:

"\u0001\u001F\u008B\b\u0000\u0000\u0000\u0000\u0000\u0004\u0000sv\nÖ\v\tÐstÔóÉOÏÌ\vI-.I-ÒK­HUPÒ\u000Fµ\n\tpLÉÍÌË,.)J,É/R\u0002\n\u0006X\u0005$ª¨\u0094\e\u0014¥\u0080x¡\u008EØÔ8[ù\*\u0001\u00004\u0088@¦Y\u0000\u0000\u0000"

So, in my input filter I receive the following error:  
[ERROR][logstash.filters.ruby] Ruby exception occurred: not in gzip format

Does anyone know how I can convert the message to the expected format, or acting on the way in wich Logstash is storing this field in the queue?

Thank you!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 24, 2019, 11:02am UTC](https://discuss.elastic.co/t/logstash-input-filter-ruby-persisted-vs-memory-queue-different-way-to-store-a-blob-column-from-oracle/209481/2 "2019-12-24T11:02:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.