Logstash plugin offline pack not persisted

I have created logstash-offline-plugins zip with all the plugins I want. I am able to install this pack using the command

bin/logstash-plugin install file:///var/logstash_offline/logstash-offline-plugins-6.7.0.zip

Then I am also able to view all the plugins with the command

bin/logstash-plugin list

But after few days working, after I restart Logstash for any logstash.conf change I have made, I see the plugin I have installed with offline plugins are removed.