# Is the multiline codec supposed to work with the JDBC input plugin?

**URL:** <https://discuss.elastic.co/t/is-the-multiline-codec-supposed-to-work-with-the-jdbc-input-plugin/268852>\
**Category:** Logstash\
**Created:** [March 31, 2021, 1:03am UTC](https://discuss.elastic.co/t/is-the-multiline-codec-supposed-to-work-with-the-jdbc-input-plugin/268852 "2021-03-31T01:03:38Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![ehanft](https://avatars.discourse-cdn.com/v4/letter/e/5f8ce5/32.png) [@ehanft](https://discuss.elastic.co/u/ehanft)\
**Post date:** [March 31, 2021, 1:03am UTC](https://discuss.elastic.co/t/is-the-multiline-codec-supposed-to-work-with-the-jdbc-input-plugin/268852/1 "2021-03-31T01:03:38Z")

</div>

I been having a hard time processing log files stored in an Oracle database table. The logs are stored as CLOB, but converted to VARCHAR2 using a pipelined table function. This works great with the JDBC input plugin in Logstash.

The problem is when I try to use the multiline codec, since Logstash seems to be ignoring it. When running the equivalent configuration, but with the file input plugin, the multiline codec works as expected. I even created a minimal scenario with 3 rows on an Oracle table, with a VARCHAR2 column, and had the same result.

Is the multiline codec supposed to work with the JDBC input plugin? Is there additional configuration required? I see it getting called in the debug log, but it doesn't really do anything...

I am using Logstash 7.11.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [March 31, 2021, 1:18am UTC](https://discuss.elastic.co/t/is-the-multiline-codec-supposed-to-work-with-the-jdbc-input-plugin/268852/2 "2021-03-31T01:18:00Z")

</div>

I do not believe the jdbc plugin uses a codec. There is a [comment](https://github.com/logstash-plugins/logstash-input-jdbc/blob/878a73eb67b68366fac74fcc65e9cadd6cf44766/lib/logstash/inputs/jdbc.rb#L132) in the code that supports this view. As the documentation for the input says, "Each row in the resultset becomes a single event."

I believe there are other inputs or outputs that ignore the codec option.

---

<div class="post-metadata">

**Author:** ![ehanft](https://avatars.discourse-cdn.com/v4/letter/e/5f8ce5/32.png) [@ehanft](https://discuss.elastic.co/u/ehanft)\
**Post date:** [March 31, 2021, 1:39am UTC](https://discuss.elastic.co/t/is-the-multiline-codec-supposed-to-work-with-the-jdbc-input-plugin/268852/3 "2021-03-31T01:39:03Z")

</div>

The jdbc plugin takes a codec in the configuration, you see it in the code that defaults to plain. I saw it in the debug as well.

Because it creates a single event per database returned row, it then looks like it really ignores the codec. Seems like something that should be configurable, or at the very least should have a warning or error in the log indicating so.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [March 31, 2021, 2:34am UTC](https://discuss.elastic.co/t/is-the-multiline-codec-supposed-to-work-with-the-jdbc-input-plugin/268852/4 "2021-03-31T02:34:30Z")

</div>

Yes, it ignores the codec. The only reason it enables the codec option is that logstash requires it. Every input and output has to have a codec option. Using it is optional.

[This](https://discuss.elastic.co/t/s3snssqs-multiline-support/266370) is another example of this, with a workaround.

---

<div class="post-metadata">

**Author:** ![ehanft](https://avatars.discourse-cdn.com/v4/letter/e/5f8ce5/32.png) [@ehanft](https://discuss.elastic.co/u/ehanft)\
**Post date:** [April 1, 2021, 9:37pm UTC](https://discuss.elastic.co/t/is-the-multiline-codec-supposed-to-work-with-the-jdbc-input-plugin/268852/5 "2021-04-01T21:37:52Z")

</div>

I tried the workaround you mentioned in your link, using the TCP input/output plugins on two separate pipelines. The output of the pipeline comes out as JSON.

Can I change the format of the TCP output plugin somehow, so configuring the multiline codec is simpler/possible?  
Or, do I need to include the JSON formatting added in the first pipeline as part of the multiline pattern?  
Or, something else?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 1, 2021, 10:06pm UTC](https://discuss.elastic.co/t/is-the-multiline-codec-supposed-to-work-with-the-jdbc-input-plugin/268852/6 "2021-04-01T22:06:08Z")

</div>

> [@ehanft](#):
>
> Can I change the format of the TCP output plugin somehow

Yes, the [default codec](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-tcp.html#plugins-outputs-tcp-codec) for a tcp output is json. You could change that to line (or possibly plain), perhaps with a format option on the codec.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 29, 2021, 10:06pm UTC](https://discuss.elastic.co/t/is-the-multiline-codec-supposed-to-work-with-the-jdbc-input-plugin/268852/7 "2021-04-29T22:06:10Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
