# How Logstash Process data

**URL:** <https://discuss.elastic.co/t/how-logstash-process-data/261216>\
**Category:** Logstash\
**Created:** [January 15, 2021, 7:42am UTC](https://discuss.elastic.co/t/how-logstash-process-data/261216 "2021-01-15T07:42:05Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![tusharnemade](https://avatars.discourse-cdn.com/v4/letter/t/67e7ee/32.png) [@tusharnemade](https://discuss.elastic.co/u/tusharnemade)\
**Post date:** [January 15, 2021, 7:42am UTC](https://discuss.elastic.co/t/how-logstash-process-data/261216/1 "2021-01-15T07:42:05Z")

</div>

Hello Team

I am in requirement of pushing data from Oracle Table to Elasticsearch Index.

My Oracle Table is of 300GB in Size.

I would like to understand , How Logstash process the data when he pulls from Oracle { input } and dumps into Elasticsearch Index {ouptut}.

Will he take whole 300GB of data into memory and process the output OR internally there is some buffer after that it pushes that much of intermediate data into ES Index.

Could someone please help me in understanding its behaviour.

This is required for me to proceed further with implementation.

Thanks  
Tushar Nemade

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [January 15, 2021, 4:30pm UTC](https://discuss.elastic.co/t/how-logstash-process-data/261216/2 "2021-01-15T16:30:34Z")

</div>

logstash will use a cursor for the result set, so that it fetches results in groups. That is mentioned [here](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-jdbc.html#_dealing_with_large_result_sets). Events are then processed through the pipeline and sent to the output in batches.

---

<div class="post-metadata">

**Author:** ![tusharnemade](https://avatars.discourse-cdn.com/v4/letter/t/67e7ee/32.png) [@tusharnemade](https://discuss.elastic.co/u/tusharnemade)\
**Post date:** [January 25, 2021, 2:49am UTC](https://discuss.elastic.co/t/how-logstash-process-data/261216/3 "2021-01-25T02:49:33Z")

</div>

Thank You Badger.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 22, 2021, 2:49am UTC](https://discuss.elastic.co/t/how-logstash-process-data/261216/4 "2021-02-22T02:49:36Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.