Filebeat nginx module does not work with AWS ES

Filebeat nginx module does not work with AWS ES due to lack of support for ingest-geoip plugin in AWS ES.

Is there any temporary workaround for this issue ? I don't need the geoip information for now.

Also any update on https://github.com/elastic/beats/issues/10867 ?

Thanks
John

Hi @john_eapen, the github issue still under discussion. We will update it with conclusion.
Meantime, maybe you can disable geoip manually by removing the processor in https://github.com/elastic/beats/blob/master/filebeat/module/nginx/access/ingest/default.json#L126 and rebuild filebeat binary.

Hi Kaiyan,

Thanks for you reply.
But this did not work for me. Can you tell me if the following is supposed to work or am I missing something?

We use docker/kubernetes. Instead of rebuilding the entire binary, i adjusted the Dockerfile as follows to include the modified "nginx/access/ingest/default.json" where i removed all references to geoip processor.

cat docker/Dockerfile
FROM docker.elastic.co/beats/filebeat-oss:7.4.1
COPY fields.yml /usr/share/filebeat/fields.yml
COPY module/nginx/access/ingest/default.json
/usr/share/filebeat/module/nginx/access/ingest/default.json #Added this line
USER root

With this docker image, I checked the running pod and the ingest file does not the geoip references.
But it did not work.

Thanks

Hmmm did you build filebeat-oss from the source code?
Maybe something like https://github.com/elastic/beats/blob/master/filebeat/tests/open-file-handlers/Dockerfile-filebeat

Hi Kaiyan,

Appreciate your help. Sorry could not reply to you earlier.

did you build filebeat-oss from the source code?
Not I did not build the entire filebeat-oss from source code. But as I mentioned earlier, I made sure that the "module/nginx/access/ingest/default.json" that is used by filebeat does not have any references to "geo-ip". I did that by including this modified default.json into my own dockerfile/build.

My question is,
Is it necessary to rebuilt filebeat-oss ? Is the nginx ingest/default.json not read from the file by filebeat at startup. ?

Thanks
John