# Logstash filter plugin development flow with SQLite

**URL:** <https://discuss.elastic.co/t/logstash-filter-plugin-development-flow-with-sqlite/89940>\
**Category:** Logstash\
**Created:** [June 19, 2017, 1:43pm UTC](https://discuss.elastic.co/t/logstash-filter-plugin-development-flow-with-sqlite/89940 "2017-06-19T13:43:19Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![nacastroc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nacastroc/32/19253_2.png) [@nacastroc](https://discuss.elastic.co/u/nacastroc)\
**Post date:** [June 19, 2017, 1:43pm UTC](https://discuss.elastic.co/t/logstash-filter-plugin-development-flow-with-sqlite/89940/1 "2017-06-19T13:43:19Z")

</div>

I'm a newbie at Logstash plugin development and, as both a work task and a training exercise, I'm developing a filter plugin that involves querying an SQLite database on each event. As of this moment, I understand the following:

> The Logstash register method is like an initialize method.

Which typically only runs once, so this is where I connect to my SQLite database and create my `@database` object as an instance variable.

> The plugin’s filter method is where the actual filtering work takes place!

So this method runs every time an event is to be processed, therefore It is here where I work with the `@database` query's `@result_set`; however, this poses the question of **where do I make sure that the database is closed?** If the `register` method is run only once, and I close the database connection at the end of the `filter` method execution, wouldn't that deny the querying of the data in the following event process, since the database is now closed?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 22, 2017, 5:24am UTC](https://discuss.elastic.co/t/logstash-filter-plugin-development-flow-with-sqlite/89940/2 "2017-06-22T05:24:32Z")

</div>

> Which typically only runs once, so this is where I connect to my SQLite database and create my @database object as an instance variable.

This only works if connections are _never_ closed during execution. I'd instead create connections whenever needed so that a dropped connection can be recovered without a Logstash restart.

> where do I make sure that the database is closed?

Is an explicit close necessary? If Logstash shuts down the socket connection will be closed by the kernel which will be picked up by the peer.

---

<div class="post-metadata">

**Author:** ![nacastroc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nacastroc/32/19253_2.png) [@nacastroc](https://discuss.elastic.co/u/nacastroc)\
**Post date:** [June 26, 2017, 12:09pm UTC](https://discuss.elastic.co/t/logstash-filter-plugin-development-flow-with-sqlite/89940/3 "2017-06-26T12:09:17Z")

</div>

> [@magnusbaeck](#):
>
> Is an explicit close necessary?

No, actually I was just trying to comply with good programming practices as it is pointed out often in most SQLite tutorials. Didn't know that it could be left to the kernel to handle. Thanks for the update. I'll test it and come back with the results.

---

<div class="post-metadata">

**Author:** ![nacastroc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nacastroc/32/19253_2.png) [@nacastroc](https://discuss.elastic.co/u/nacastroc)\
**Post date:** [June 30, 2017, 12:41pm UTC](https://discuss.elastic.co/t/logstash-filter-plugin-development-flow-with-sqlite/89940/4 "2017-06-30T12:41:39Z")

</div>

Hello again! Already checked your approach and it solves my initial problem. Now I'm trying to work on performance but, since that is out of the scope of my original question, I'm marking this as the solution. Thanks a lot.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 28, 2017, 12:41pm UTC](https://discuss.elastic.co/t/logstash-filter-plugin-development-flow-with-sqlite/89940/5 "2017-07-28T12:41:40Z")

</div>

Closed on July 28, 2017, 12:41pm UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
