Using Filebeat with AWS ES with ingest-geoip disabled

[chiemeleakoma]

I have setup filebeat with basic config as a proof of concept. After starting the service, i couldnt see any logs ingested by ES, but filebeat error log shows:

2019-11-26T15:56:28.174Z ERROR pipeline/output.go:100 Failed to connect to backoff(elasticsearch(http://xxx.eu-west-1.es.amazonaws.com:80)): Connection marked as failed because the onConnect callback failed: Error loading pipeline for fileset iis/error: This module requires the following Elasticsearch plugins: ingest-geoip. You can install them by running the following commands on all the Elasticsearch nodes:
sudo bin/elasticsearch-plugin install ingest-geoip.

From AWS ES specs, the ingest-geoip plugin is not supported. Is it possible to have filebeat running but with ingest-geoip disabled.

could be an optional feature, rather than a requirement.

[Kaiyan_Sheng]

Thanks @chiemeleakoma! I saw you just commented on the github issue https://github.com/elastic/beats/issues/10867. We will triage it and prioritize it soon!

[Christian_Dahlqvist]

Removing this plugin will mean that no location information will be derived from the IP information in the access logs, which is often very useful information. This will affect the usefulness of the dashboards as well. I would recommend using Elastic's Elasticsearch service instead as it allows this plugin (and a lot of other features) and offers a free trial period.