Filebeat issues - geoip installation error

pge · June 29, 2018, 7:03am

Hi all,
on centos 7.5 I have to test filebeat 6.3.0. After rhe installation i have problems with geoip plugin.

On filebeat log i have this message:

2018-06-28T16:23:04.673+0200    ERROR   pipeline/output.go:74   Failed to connect: Connection marked as failed because the onConnect callback failed: Error loading pipeline for fileset system/auth: This module requires the ingest-geoip plugin to be installed in Elasticsearch. You can install it using the following command in the Elasticsearch home directory:
        sudo bin/elasticsearch-plugin install ingest-geoip

-- Test 1

cd /usr/share/elasticsearch

bin/elasticsearch-plugin install ingest-geoip

Exception in thread "main" java.net.ConnectException: Connection timed out (Connection timed out)
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:589)
        at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:673)
        at sun.security.ssl.BaseSSLSocketImpl.connect(BaseSSLSocketImpl.java:173)
        at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
        at sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:264)
        at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:367)
        at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191)
        at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1156)
        at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1050)
        at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177)
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:162)
        at org.elasticsearch.plugins.InstallPluginCommand.urlExists(InstallPluginCommand.java:325)
        at org.elasticsearch.plugins.InstallPluginCommand.getElasticUrl(InstallPluginCommand.java:292)
        at org.elasticsearch.plugins.InstallPluginCommand.download(InstallPluginCommand.java:248)
        at org.elasticsearch.plugins.InstallPluginCommand.execute(InstallPluginCommand.java:223)
        at org.elasticsearch.plugins.InstallPluginCommand.execute(InstallPluginCommand.java:210)
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124)
        at org.elasticsearch.cli.MultiCommand.execute(MultiCommand.java:79)
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124)
        at org.elasticsearch.cli.Command.main(Command.java:90)

-- Test 2

export http_proxy=http://myproxyip:myproxyport
export https_proxy=http://myproxyip:myproxyport

ES_JAVA_OPTS="-Dhttp.proxyHost=<myproxyip> -Dhttp.proxyPort=<myproxyport> -Dhttps.proxyHost=<myproxyip> -Dhttps.proxyPort=<myproxyport>" bin/elasticsearch-plugin install ingest-geoip

-same java exception of test 1

-- Test 3
Over network doesan work, then I have tested the offline installation:

bin/elasticsearch-plugin install ingest-geoip-6.3.0.zip
/usr/share/elasticsearch/bin/elasticsearch-plugin -v install /usr/share/elasticsearch/ingest-geoip-6.3.0.zip

 tool for managing installed elasticsearch plugins

Commands
--------
list - Lists installed elasticsearch plugins
install - Install a plugin
remove - removes a plugin from Elasticsearch

Non-option arguments:
command              

Option         Description        
------         -----------        
-h, --help     show help          
-s, --silent   show minimal output
-v, --verbose  show verbose output
ERROR: Unknown plugin ingest-geoip-6.3.0.zip

--- Test 4

sudo yum list "*geoip*"
Loaded plugins: etckeeper, fastestmirror
Loading mirror speeds from cached hostfile

Installed Packages
GeoIP.x86_64                    1.5.0-11.el7    @anaconda           

Available Packages             
GeoIP.i686                      1.5.0-11.el7    centos7-x86_64      
GeoIP-data.noarch               1.5.0-11.el7    centos7-x86_64      
GeoIP-devel.i686                1.5.0-11.el7    centos7-x86_64      
GeoIP-devel.x86_64              1.5.0-11.el7    centos7-x86_64      
GeoIP-update.noarch             1.5.0-11.el7    centos7-x86_64      
lighttpd-mod_geoip.x86_64       1.4.49-1.el7    epel7-centos7-x86_64
mod_geoip.x86_64                1.2.10-1.el7    epel7-centos7-x86_64
nginx-mod-http-geoip.x86_64     1:1.12.2-2.el7  epel7-centos7-x86_64
opensips-mmgeoip.x86_64         1.10.5-3.el7    epel7-centos7-x86_64
pdns-backend-geoip.x86_64       3.4.11-4.el7    epel7-centos7-x86_64
php-pecl-geoip.x86_64           1.0.8-5.el7     epel7-centos7-x86_64
python-GeoIP.x86_64             1.2.8-5.el7     epel7-centos7-x86_64
python-pygeoip.noarch           0.2.6-5.el7     epel7-centos7-x86_64
syslog-ng-geoip.x86_64          3.5.6-3.el7     epel7-centos7-x86_64
uwsgi-plugin-geoip.x86_64       2.0.16-1.el7i	epel7-centos7-x86_64

any idea?

pierhugues · June 29, 2018, 1:13pm

Hello @pge,

To do an offline installation of the plugin you have to give the complete path to the file.

Something like this:

bin/elasticsearch-plugin install file:///path/to/plugin.zip

From your messages, I think it will be the following:

bin/elasticsearch-plugin install file:///usr/share/elasticsearch/ingest-geoip-6.3.0.zip

pge · July 17, 2018, 6:23am

Hi @pierhugues,
Thanks a lot!
I have added the file:// to the command line and now the installation process start.

What I cannot understand is why I have a security warning. I have checked the link to the Oracle site and seems that I have to add a permission role to the plugin (I´have launched the command as root)....

cd /usr/share/elasticsearch
bin/elasticsearch-plugin install -v file:///tmp/ingest-geoip-6.3.0.zip

-> Downloading file:///tmp/ingest-geoip-6.3.0.zip
Retrieving zip from file:///tmp/ingest-geoip-6.3.0.zip
[=================================================] 100%   
- Plugin information:
Name: ingest-geoip
Description: Ingest processor that uses looksup geo data based on ip adresses using the Maxmind geo database
Version: 6.3.0
Elasticsearch Version: 6.3.0
Java Version: 1.8
Native Controller: false
Extended Plugins: []
 * Classname: org.elasticsearch.ingest.geoip.IngestGeoIpPlugin
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@     WARNING: plugin requires additional permissions     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.lang.RuntimePermission accessDeclaredMembers
* java.lang.reflect.ReflectPermission suppressAccessChecks
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks.

Continue with installation? [y/N]y

pierhugues · August 2, 2018, 1:09pm

@pge We sandbox what elasticsearch can do when we start it, some plugins requires more permissions that what elasticsearch support, using 'y' will make elasticsearch add theses permissions.

pge · August 2, 2018, 2:21pm

Thanks a lot for your help @pierhugues and your courtesy.
I solved my problem with your suggestions!

system · August 30, 2018, 2:21pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.