Hi,
My test server does not have access to the internet, and when I try to install plugin locally, they fail. I see a fair amount of issues with this, but nothing showing a solution. Is there a way to do this?
i try many variations of the following, and they all fail.
./logstash-plugin --local install /tmp/logstash-codec-sflow-2.0.1.gem
./logstash-plugin --no-verify install /tmp/logstash-codec-sflow-2.0.1.gem
Do you get any errors?
Thanks
If I try it without --local, it try to access rubygems website. If I try it with local, I get "unrecognized option '--local'".
What version of logstash?
6.1.3 on CentOS 7.4
Unfortunately this is a 3rd party plugin repo. The author has pinned the Logstash Core version in the gemspec file.
s.add_runtime_dependency 'logstash-core', '>= 5.4.0', '<= 5.4.1'
^ the above means that the author states `version 2.0.1 can only work with Logstash (Core) v5.4.0 or v5.4.1.
You may be able to ask the author to lift this restriction by removing that line - the part 'logstash-core-plugin-api', '>= 1.60', '<= 2.99' is designed to constrain plugin version against LS core versions.
Thanks. So, native logstash doesn't provide an ability to install plug-ins offline?
OK. So, you're saying the plugin is preventing it from being installed. I look at the logstash-codec-netflow plugin, and see no such limitation, but it fails with the same errors.
You will need to download version 2.0.0 of the gem for an offline install. Interestingly this is the same version you get when you do an online install. That 2.0.1 release clearly has issues. It doesn't look like there was anything added in the 2.0.1 commit that would be a must-have. Just stick to 2.0.0. I have been using it over a year without issue, including with all of the 5.x and 6.x releases of Logstash.
Tried installing it locally again - now I get the following:
[root@elk-test bin]# ./logstash-plugin install /tmp/logstash-codec-sflow-2.0.1.gem
WARNING: A maven settings file already exist at /root/.m2/settings.xml, please review the content to make sure it include your proxies configuration.
ERROR: Something went wrong when installing /tmp/logstash-codec-sflow-2.0.1.gem, message: certificate verify failed
This shouldn't be this difficult.
Just stick to 2.0.0. The 2.0.1 release has issues, but 2.0.0 works fine.
Thanks Rob.
Yeah, I tried that. The plugin installer always tries to access the internet, no matter what, which is just… ridiculous. From what I can gather on my searches, this is a common problem, and ELK doesn’t have a way to install offline plugins (which, again, is ridiculous).
[root@elk-test bin]# ./logstash-plugin install /tmp/logstash-codec-sflow-2.0.0.gem
ERROR: Something went wrong when installing /tmp/logstash-codec-sflow-2.0.0.gem, message: certificate verify failed
[root@elk-test bin]# ./logstash-plugin install /tmp/logstash-codec-sflow-2.0.0.gem --local
ERROR: Something went wrong when installing /tmp/logstash-codec-sflow-2.0.0.gem, --local, message: certificate verify failed
[root@elk-test bin]# ./logstash-plugin install /tmp/logstash-codec-sflow-2.0.0.gem --no-verify
ERROR: Something went wrong when installing /tmp/logstash-codec-sflow-2.0.0.gem, --no-verify, message: certificate verify failed
[root@elk-test bin]#
So, essentially, there is no way to install plugins without having internet access. Ugh. Major deficiency. Can't believe this isn't an option.
Well, the docs state that you would need two machines, 1 online and 1 offline. Prepare the pack on the online machine then copy the pack over to the offline machine and install it. You have to have some access to the offline machine in order to copy the gem file to it surely.
A major problem with using the gem file method is that, if the gem has dependencies it needs it will try to download them and fail. This is not a Logstash problem per se, this is how the Ruby Gem ecosystem works.
We switched to using the offline pack method because the online install will install deps on the local machine and then they are available in the pack.
PS. you need not be so disparaging - you are using a free-of-charge open source project that is using other FOC OS projects to do a lot of the infrastructure stuff like managing plugins, talking to kafka, netflow, jdbc, AWS S3 etc.
Thanks. Don't mean to be disparaging, but I believe in being honest. I use plenty of other FOSS/COTS tools, and if they are missing functionality, I tell them as well. If you don't mention it, or ask, it will never get provided.
One of the other ElastiFlow users was also having an issue with the sFlow codec. After a closer look, my initial assessment was wrong. There was a useful fix in 2.0.1 that adds error handling of malformed sFlow packets. Since 2.0.1 can't be used with later Logstash versions, I submited a PR to the codec's owner that loosens the version dependency for logstash-core. Until he merges the PR and creates a release I have an installable gem available from my fork here...
To use this, first remove the previous version and then install the new one using these commands...
LS_HOME/bin/logstash-plugin remove logstash-codec-sflow
LS_HOME/bin/logstash-plugin install /PATH/TO/logstash-codec-sflow-2.0.2.gem
Please let me know how this works for you.
(sorry @bossi6of9 you probably still need an active internet connection)
If you try this fix, please let me know how it works for you.
Rob
Updating this thread for future readers...
The maintainer of the sFlow codec has merged my PR, and released 2.0.2 to rubygems.org. With the dependencies corrected I can confirm that 2.0.2 will be fetched using an online install.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.