# Jdbc input for oracle blob field error

**URL:** https://discuss.elastic.co/t/jdbc-input-for-oracle-blob-field-error/139861
**Category:** Logstash
**Created:** [July 13, 2018, 2:58am UTC](https://discuss.elastic.co/t/jdbc-input-for-oracle-blob-field-error/139861 "2018-07-13T02:58:38Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![Yosen](https://avatars.discourse-cdn.com/v4/letter/y/41988e/32.png) [@Yosen](https://discuss.elastic.co/u/Yosen)
#### Post date: [July 13, 2018, 2:58am UTC](https://discuss.elastic.co/t/jdbc-input-for-oracle-blob-field-error/139861/1 "2018-07-13T02:58:38Z")

</div>

Dear experts:

I'm using the jdbc input to load data from a oracle table to ES. There is a blob field in the table. If the blob field only have text content, it can transfer to ES. But when this filed have some binary data, such as PDF file, it will failed. The error log as below:

[2018-07-12T17:56:54,720][ERROR][logstash.pipeline] A plugin had an unrecoverable error. Will restart this plugin.  
Pipeline\_id:main  
Plugin: \<LogStash::Inputs::Jdbc jdbc\_connection\_string=\>"jdbc:oracle:thin:@192.168.0.1:1521:oracle", jdbc\_user=\>"oracle", jdbc\_password=\>, jdbc\_driver\_library=\>"/data/logstash/ojdbc7.jar", jdbc\_driver\_class=\>"Java::oracle.jdbc.driver.OracleDriver", columns\_charset=\>{"msg\_bytes"=\>"UTF-8"}, record\_last\_run=\>true, use\_column\_value=\>false, tracking\_column=\>"LOG\_TIME", last\_run\_metadata\_path=\>"/data/logstash/logstash-6.3.1/last\_run", clean\_run=\>false, jdbc\_paging\_enabled=\>true, jdbc\_page\_size=\>50000, jdbc\_fetch\_size=\>5000, jdbc\_validate\_connection=\>true, connection\_retry\_attempts=\>5, statement\_filepath=\>"/data/logstash/jdbc\_test.sql", add\_field=\>{"key\_id"=\>"%{msg\_id}%-%{direction}-%{log\_location}", "log\_bytes"=\>""}, type=\>"jdbc", id=\>"cbe228beb0a0f45de90c1aa4976decf0effd7c4a323fe0f7f209ffd03f0d1d2e", enable\_metric=\>true, codec=\>\<LogStash::Codecs::Plain id=\>"plain\_4678de54-517f-4429-ae4f-2e6f34b5cc74", enable\_metric=\>true, charset=\>"UTF-8"\>, jdbc\_validation\_timeout=\>3600, jdbc\_pool\_timeout=\>5, sql\_log\_level=\>"info", connection\_retry\_attempts\_wait\_time=\>0.5, parameters=\>{"sql\_last\_value"=\>2018-07-12 17:51:16 +0800}, tracking\_column\_type=\>"numeric", lowercase\_column\_names=\>true\>  
Error: private method `warn' called for nil:NilClass Exception: NoMethodError Stack: /data/logstash/logstash-6.3.1/logstash-core/lib/logstash/util/charset.rb:28:in`block in convert'  
org/jruby/RubyKernel.java:1741:in `tap' /data/logstash/logstash-6.3.1/logstash-core/lib/logstash/util/charset.rb:27:in`convert'  
/data/logstash/logstash-6.3.1/vendor/bundle/jruby/2.3.0/gems/logstash-input-jdbc-4.3.9/lib/logstash/inputs/jdbc.rb:288:in `convert' /data/logstash/logstash-6.3.1/vendor/bundle/jruby/2.3.0/gems/logstash-input-jdbc-4.3.9/lib/logstash/inputs/jdbc.rb:267:in`block in execute\_query'  
org/jruby/RubyHash.java:1343:in `each' org/jruby/RubyEnumerable.java:830:in`map'  
/data/logstash/logstash-6.3.1/vendor/bundle/jruby/2.3.0/gems/logstash-input-jdbc-4.3.9/lib/logstash/inputs/jdbc.rb:267:in `block in execute_query' /data/logstash/logstash-6.3.1/vendor/bundle/jruby/2.3.0/gems/logstash-input-jdbc-4.3.9/lib/logstash/plugin_mixins/jdbc.rb:231:in`block in execute\_statement'  
/data/logstash/logstash-6.3.1/vendor/bundle/jruby/2.3.0/gems/logstash-input-jdbc-4.3.9/lib/logstash/plugin\_mixins/jdbc.rb:253:in `block in perform_query' /data/logstash/logstash-6.3.1/vendor/bundle/jruby/2.3.0/gems/sequel-5.9.0/lib/sequel/dataset/actions.rb:151:in`block in each'  
/data/logstash/logstash-6.3.1/vendor/bundle/jruby/2.3.0/gems/sequel-5.9.0/lib/sequel/adapters/jdbc.rb:795:in `process_result_set' /data/logstash/logstash-6.3.1/vendor/bundle/jruby/2.3.0/gems/sequel-5.9.0/lib/sequel/adapters/jdbc.rb:726:in`block in fetch\_rows'  
/data/logstash/logstash-6.3.1/vendor/bundle/jruby/2.3.0/gems/sequel-5.9.0/lib/sequel/adapters/jdbc.rb:244:in `block in execute' /data/logstash/logstash-6.3.1/vendor/bundle/jruby/2.3.0/gems/sequel-5.9.0/lib/sequel/adapters/jdbc.rb:675:in`statement'  
/data/logstash/logstash-6.3.1/vendor/bundle/jruby/2.3.0/gems/sequel-5.9.0/lib/sequel/adapters/jdbc.rb:239:in `block in execute' /data/logstash/logstash-6.3.1/vendor/bundle/jruby/2.3.0/gems/sequel-5.9.0/lib/sequel/connection_pool/threaded.rb:91:in`hold'  
/data/logstash/logstash-6.3.1/vendor/bundle/jruby/2.3.0/gems/sequel-5.9.0/lib/sequel/database/connecting.rb:270:in `synchronize' /data/logstash/logstash-6.3.1/vendor/bundle/jruby/2.3.0/gems/sequel-5.9.0/lib/sequel/adapters/jdbc.rb:238:in`execute'  
/data/logstash/logstash-6.3.1/vendor/bundle/jruby/2.3.0/gems/sequel-5.9.0/lib/sequel/dataset/actions.rb:1082:in `execute' /data/logstash/logstash-6.3.1/vendor/bundle/jruby/2.3.0/gems/sequel-5.9.0/lib/sequel/adapters/jdbc.rb:726:in`fetch\_rows'  
/data/logstash/logstash-6.3.1/vendor/bundle/jruby/2.3.0/gems/sequel-5.9.0/lib/sequel/dataset/actions.rb:151:in `each' /data/logstash/logstash-6.3.1/vendor/bundle/jruby/2.3.0/gems/logstash-input-jdbc-4.3.9/lib/logstash/plugin_mixins/jdbc.rb:252:in`block in perform\_query'  
/data/logstash/logstash-6.3.1/vendor/bundle/jruby/2.3.0/gems/sequel-5.9.0/lib/sequel/extensions/pagination.rb:60:in `block in each_page' org/jruby/RubyRange.java:485:in`each'  
/data/logstash/logstash-6.3.1/vendor/bundle/jruby/2.3.0/gems/sequel-5.9.0/lib/sequel/extensions/pagination.rb:60:in `each_page' /data/logstash/logstash-6.3.1/vendor/bundle/jruby/2.3.0/gems/logstash-input-jdbc-4.3.9/lib/logstash/plugin_mixins/jdbc.rb:251:in`perform\_query'  
/data/logstash/logstash-6.3.1/vendor/bundle/jruby/2.3.0/gems/logstash-input-jdbc-4.3.9/lib/logstash/plugin\_mixins/jdbc.rb:229:in `execute_statement' /data/logstash/logstash-6.3.1/vendor/bundle/jruby/2.3.0/gems/logstash-input-jdbc-4.3.9/lib/logstash/inputs/jdbc.rb:264:in`execute\_query'  
/data/logstash/logstash-6.3.1/vendor/bundle/jruby/2.3.0/gems/logstash-input-jdbc-4.3.9/lib/logstash/inputs/jdbc.rb:250:in `run' /data/logstash/logstash-6.3.1/logstash-core/lib/logstash/pipeline.rb:512:in`inputworker'  
/data/logstash/logstash-6.3.1/logstash-core/lib/logstash/pipeline.rb:505:in `block in start\_input'

Who can kindly help me to solve this problem?  
Or who can tell me how to configure it so that it can continue on error, not retry on error.

---

<div class="post-metadata">

### Author: ![Yosen](https://avatars.discourse-cdn.com/v4/letter/y/41988e/32.png) [@Yosen](https://discuss.elastic.co/u/Yosen)
#### Post date: [July 16, 2018, 12:43am UTC](https://discuss.elastic.co/t/jdbc-input-for-oracle-blob-field-error/139861/2 "2018-07-16T00:43:47Z")

</div>

Who can help me？

---

<div class="post-metadata">

### Author: ![Yosen](https://avatars.discourse-cdn.com/v4/letter/y/41988e/32.png) [@Yosen](https://discuss.elastic.co/u/Yosen)
#### Post date: [July 19, 2018, 12:30am UTC](https://discuss.elastic.co/t/jdbc-input-for-oracle-blob-field-error/139861/3 "2018-07-19T00:30:58Z")

</div>

who can give me some suggestion?

---

<div class="post-metadata">

### Author: ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)
#### Post date: [July 19, 2018, 6:42am UTC](https://discuss.elastic.co/t/jdbc-input-for-oracle-blob-field-error/139861/4 "2018-07-19T06:42:15Z")

</div>

Argh. I've seen this before, and I'll dig into it in the morning. IIRC, the charset converter needs to be connected to the plugin's logger, and can fail in this way if not implemented correctly in the plugin. Are you using the latest jdbc plugin?

```auto
${LOGSTASH_HOME}/bin/logstash-plugin update logstash-input-jdbc

```

---

<div class="post-metadata">

### Author: ![Yosen](https://avatars.discourse-cdn.com/v4/letter/y/41988e/32.png) [@Yosen](https://discuss.elastic.co/u/Yosen)
#### Post date: [July 20, 2018, 2:32am UTC](https://discuss.elastic.co/t/jdbc-input-for-oracle-blob-field-error/139861/5 "2018-07-20T02:32:41Z")

</div>

> [@yaauie](#):
>
> /bin/logstash-plugin update logstash-input-jdbc

Hi, yaauie:

Thank you for your help. I will run the update command and test it again. But I remember it is the latest.  
I don't know ruby language， but from the error message, i find it seems that the jdbc plugin call the "charset.rb" method "convert", maybe this raise the error. That mean is not because of the jdbc plugin, it because of the logstash core code.

Yosen

---

<div class="post-metadata">

### Author: ![Yosen](https://avatars.discourse-cdn.com/v4/letter/y/41988e/32.png) [@Yosen](https://discuss.elastic.co/u/Yosen)
#### Post date: [July 20, 2018, 5:49am UTC](https://discuss.elastic.co/t/jdbc-input-for-oracle-blob-field-error/139861/6 "2018-07-20T05:49:10Z")

</div>

Hi，yaauie：

when I update the jdbc plugin, there is an error:  
Updating logstash-input-jdbc  
Error Bundler::InstallError, retrying 1/10  
An error occurred while installing manticore (0.6.4), and Bundler cannot continue.  
Make sure that `gem install manticore -v '0.6.4'` succeeds before bundling.  
No plugin updated

Is it because of logstash version? My logstash version is 6.3.1  
And I found the latest jdbc plugin version is 4.3.9, I don't know how to get the plugin version in my logstash. But In the path /logstash-6.3.1/vendor/bundle/jruby/2.3.0/gems, I can find a folder logstash-input-jdbc-4.3.9, so i think i use the latest version.

---

<div class="post-metadata">

### Author: ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)
#### Post date: [July 20, 2018, 5:57pm UTC](https://discuss.elastic.co/t/jdbc-input-for-oracle-blob-field-error/139861/7 "2018-07-20T17:57:16Z")

</div>

I've pushed a fix to the codec, but am waiting for a teammate to review before we publish the updated Gem.

If you want, you can edit your local gem to apply the patch manually, which will circumvent the bundler issues (we can chase those down in a separate thread if they continue to get in the way).

- change into the current plugin's directory, likely something like:

```auto
cd ${LOGSTASH_HOME}/vendor/bundle/jruby/2.3.0/gems/logstash-input-jdbc-4.3.9

```

- download the `.patch` from [logstash-plugins/logstash-input-jdbc#291](https://github.com/logstash-plugins/logstash-input-jdbc/pull/291):

```auto
wget https://patch-diff.githubusercontent.com/raw/logstash-plugins/logstash-input-jdbc/pull/291.patch

```

- inspect the patch that you downloaded, to make sure it's what you expect:

```auto
cat 291.patch

```

- apply the patch, stripping the git prefix and backing up the original file:

```auto
patch --strip 1 --backup < 291.patch

```

---

<div class="post-metadata">

### Author: ![Yosen](https://avatars.discourse-cdn.com/v4/letter/y/41988e/32.png) [@Yosen](https://discuss.elastic.co/u/Yosen)
#### Post date: [July 23, 2018, 1:01am UTC](https://discuss.elastic.co/t/jdbc-input-for-oracle-blob-field-error/139861/8 "2018-07-23T01:01:37Z")

</div>

Hi, yaauie:

Thank you very much. I have test it, it works. Thank you very much.  
In fact ,before you update it, I have modify the jdbc.rb file, I only add a rescue statement to catch the exception. When the exception happen, it will return a message which will tell me the conversion is failed. It works, too. I find in some plugins, it provide some failure tag function. I think it is a good idea, which can let the end user know which data has some problems. So I think maybe this jdbc plugin can implement it. It just a suggestion for you and your team .

Thank you for your help again.

Yosen

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 20, 2018, 1:01am UTC](https://discuss.elastic.co/t/jdbc-input-for-oracle-blob-field-error/139861/9 "2018-08-20T01:01:51Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
