# Plugin with in-memory dictionary for Logstash **URL:** \ **Category:** Logstash\ **Created:** [July 4, 2019, 8:41am UTC](https://discuss.elastic.co/t/plugin-with-in-memory-dictionary-for-logstash/188863 "2019-07-04T08:41:28Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![mdobrin](https://avatars.discourse-cdn.com/v4/letter/m/f0a364/32.png) [@mdobrin](https://discuss.elastic.co/u/mdobrin)\ **Post date:** [July 4, 2019, 8:41am UTC](https://discuss.elastic.co/t/plugin-with-in-memory-dictionary-for-logstash/188863/1 "2019-07-04T08:41:28Z")
Currently, my Logstash application reads data from a Kafka stream. I would like to add data from a different source which is updated once a week. To do so, I would like to have an external (Ruby or Java?) plugin which is able to load data from the new source into memory on a daily basis. (I would like to avoid querying the database everytime a Kafka message is received). Once loaded into memory, I plan on taking adding some new fields into ES which are not obtainable from Kafka. Is it possible to create a Logstash in-memory plugin which does what I describe above? If not, is there a workaround? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [July 4, 2019, 8:45am UTC](https://discuss.elastic.co/t/plugin-with-in-memory-dictionary-for-logstash/188863/2 "2019-07-04T08:45:37Z")
Does the translate or jdbc\_static plugins work for you? ---
**Author:** ![mdobrin](https://avatars.discourse-cdn.com/v4/letter/m/f0a364/32.png) [@mdobrin](https://discuss.elastic.co/u/mdobrin)\ **Post date:** [July 4, 2019, 2:02pm UTC](https://discuss.elastic.co/t/plugin-with-in-memory-dictionary-for-logstash/188863/3 "2019-07-04T14:02:59Z")
Thanks! The jdbc\_static plugin looks promising. In my case, this is not a direct connection a DB, but rather, an API is wrapped around the DB which accepts SQL-like queries. The value returned by the API to me is data represented by a Thrift object. So, I would need to be able 1) to query the API and 2) decode the Thrift objects with the jdbc\_static plugin. Do you think this is possible? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [July 4, 2019, 6:24pm UTC](https://discuss.elastic.co/t/plugin-with-in-memory-dictionary-for-logstash/188863/4 "2019-07-04T18:24:05Z")
It sounds to me like that may require a custom plugin or a script to handle the conversion. ---
**Author:** ![mdobrin](https://avatars.discourse-cdn.com/v4/letter/m/f0a364/32.png) [@mdobrin](https://discuss.elastic.co/u/mdobrin)\ **Post date:** [July 5, 2019, 1:04pm UTC](https://discuss.elastic.co/t/plugin-with-in-memory-dictionary-for-logstash/188863/5 "2019-07-05T13:04:12Z")
OK, thanks - I'll look into that. ---
**Author:** ![mdobrin](https://avatars.discourse-cdn.com/v4/letter/m/f0a364/32.png) [@mdobrin](https://discuss.elastic.co/u/mdobrin)\ **Post date:** [July 9, 2019, 4:12am UTC](https://discuss.elastic.co/t/plugin-with-in-memory-dictionary-for-logstash/188863/6 "2019-07-09T04:12:33Z")
@Christian_Dahlqvist Just to confirm, excluding the Thrift conversion specification, would the Rest API query part be supported by jdbc\_static? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [July 9, 2019, 5:18am UTC](https://discuss.elastic.co/t/plugin-with-in-memory-dictionary-for-logstash/188863/7 "2019-07-09T05:18:39Z")
No, it expects to read from a database. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 6, 2019, 5:18am UTC](https://discuss.elastic.co/t/plugin-with-in-memory-dictionary-for-logstash/188863/8 "2019-08-06T05:18:40Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.