# Is Beats mandatory to use, while using log stash JDBC plug in?

**URL:** <https://discuss.elastic.co/t/is-beats-mandatory-to-use-while-using-log-stash-jdbc-plug-in/137727>\
**Category:** Beats\
**Created:** [June 28, 2018, 3:31am UTC](https://discuss.elastic.co/t/is-beats-mandatory-to-use-while-using-log-stash-jdbc-plug-in/137727 "2018-06-28T03:31:24Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Preethi](https://avatars.discourse-cdn.com/v4/letter/p/f04885/32.png) [@Preethi](https://discuss.elastic.co/u/Preethi)\
**Post date:** [June 28, 2018, 3:31am UTC](https://discuss.elastic.co/t/is-beats-mandatory-to-use-while-using-log-stash-jdbc-plug-in/137727/1 "2018-06-28T03:31:24Z")

</div>

Hi,  
When I implement ELK for a large scale application, Do I need to include the Beats to take data from RDBS and pass through Kafka, Then logstash will come to act?  
I want to know what are the components needs to be used, Since want to integrate with a J2Ee application which currently uses an RDBMS.

Thanks,  
Preethi

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [June 28, 2018, 5:43pm UTC](https://discuss.elastic.co/t/is-beats-mandatory-to-use-while-using-log-stash-jdbc-plug-in/137727/2 "2018-06-28T17:43:44Z")

</div>

None of the Beats managed by Elastic support JDBC, but Logstash has a [jdbc input](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-jdbc.html).

So you can use Logstash with the JDBC input to pull from the DB, and then use the [Kafka output](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-kafka.html) to write the data to Kafka. In the middle you can use [filters](https://www.elastic.co/guide/en/logstash/current/filter-plugins.html) if you need to transform the data at all.

No Beats required for this.

---

<div class="post-metadata">

**Author:** ![Preethi](https://avatars.discourse-cdn.com/v4/letter/p/f04885/32.png) [@Preethi](https://discuss.elastic.co/u/Preethi)\
**Post date:** [June 29, 2018, 1:37am UTC](https://discuss.elastic.co/t/is-beats-mandatory-to-use-while-using-log-stash-jdbc-plug-in/137727/3 "2018-06-29T01:37:39Z")

</div>

Thanks for the reply....

I have one more query. Planning to use ELK for open text searching. Since Kafka is for stream processing, Does it required for offline data? I mean If the log stash .conf will be scheduled at certain interval.

Thanks

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [June 29, 2018, 1:04pm UTC](https://discuss.elastic.co/t/is-beats-mandatory-to-use-while-using-log-stash-jdbc-plug-in/137727/4 "2018-06-29T13:04:11Z")

</div>

I don't understand the question. Perhaps you could re-phrase it?

---

<div class="post-metadata">

**Author:** ![Preethi](https://avatars.discourse-cdn.com/v4/letter/p/f04885/32.png) [@Preethi](https://discuss.elastic.co/u/Preethi)\
**Post date:** [July 2, 2018, 6:23am UTC](https://discuss.elastic.co/t/is-beats-mandatory-to-use-while-using-log-stash-jdbc-plug-in/137727/5 "2018-07-02T06:23:14Z")

</div>

1. Kafka needs to be used for reducing the latency for loading offline data? Or no need to use? Is it useful for real time data streaming only?

2. To load offline data from an RDBMS, need to schedule the log stash .conf file. Is it?

Thanks.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 30, 2018, 8:23am UTC](https://discuss.elastic.co/t/is-beats-mandatory-to-use-while-using-log-stash-jdbc-plug-in/137727/6 "2018-07-30T08:23:16Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
