Disabling geoip downloader is not working (elastic 7.16.2 and java 11)

Ok... Looking Close...

Ok I downloaded a Fresh 7.16.2 and only added 1 line

ingest.geoip.downloader.enabled: false

You will still see the initialization of the "default" database upon startup BUT there is no actual geoip database loaded into elasticsearch... which you already actually confirmed...(see below)

You will see these logs lines.. which I think are a bit misleading....

[2022-04-16T19:45:41,128][INFO ][o.e.i.g.ConfigDatabases  ] [hyperion] initialized default databases [[GeoLite2-Country.mmdb, GeoLite2-City.mmdb, GeoLite2-ASN.mmdb]], config databases [[]] and watching [/Users/sbrown/workspace/elastic-install/7.16.2/tmp/elasticsearch-7.16.2/config/ingest-geoip] for changes
[2022-04-16T19:45:41,130][INFO ][o.e.i.g.DatabaseNodeService] [hyperion] initialized database registry, using geoip-databases directory [/var/folders/p9/r3gjl2912ybd827msztppr_m0000gn/T/elasticsearch-6246939774672860089/geoip-databases/C7v7WTzGQ7CnHKoZ8sbmAw]

If you then run....

hyperion:elasticsearch-7.16.2 sbrown$ curl http://127.0.0.1:9200/_cat/indices/.geo*?v
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size

That means there is no .geo ip indexes / databases are actually loaded into elasticsearch ... i.e. they are not loaded INTO elasticsearch.

This also confirms it... the geoip stats show no downloads.

hyperion:elasticsearch-7.16.2 sbrown$ curl localhost:9200/_ingest/geoip/stats
{"stats":{"successful_downloads":0,"failed_downloads":0,"total_download_time":0,"databases_count":0,"skipped_updates":0,"expired_databases":0},"nodes":{}}hyperion:elasticsearch-7.16.2

Now lets enabled it and start again...
ingest.geoip.downloader.enabled: true

You will see logs like this.....notr the updated and downloaded databases

[2022-04-16T19:50:11,468][INFO ][o.e.i.g.DatabaseNodeService] [hyperion] successfully reloaded changed geoip database file [/var/folders/p9/r3gjl2912ybd827msztppr_m0000gn/T/elasticsearch-15885661857466451462/geoip-databases/C7v7WTzGQ7CnHKoZ8sbmAw/GeoLite2-ASN.mmdb]
[2022-04-16T19:50:15,055][INFO ][o.e.i.g.DatabaseNodeService] [hyperion] downloading geoip database [GeoLite2-City.mmdb] to [/var/folders/p9/r3gjl2912ybd827msztppr_m0000gn/T/elasticsearch-15885661857466451462/geoip-databases/C7v7WTzGQ7CnHKoZ8sbmAw/GeoLite2-City.mmdb.tmp.gz]
[2022-04-16T19:50:15,080][INFO ][o.e.i.g.GeoIpDownloader  ] [hyperion] updated geoip database [GeoLite2-City.mmdb]
[2022-04-16T19:50:15,081][INFO ][o.e.i.g.GeoIpDownloader  ] [hyperion] updating geoip database [GeoLite2-Country.mmdb]
[2022-04-16T19:50:16,373][INFO ][o.e.i.g.DatabaseNodeService] [hyperion] downloading geoip database [GeoLite2-Country.mmdb] to [/var/folders/p9/r3gjl2912ybd827msztppr_m0000gn/T/elasticsearch-15885661857466451462/geoip-databases/C7v7WTzGQ7CnHKoZ8sbmAw/GeoLite2-Country.mmdb.tmp.gz]
[2022-04-16T19:50:16,387][INFO ][o.e.i.g.GeoIpDownloader  ] [hyperion] updated geoip database [GeoLite2-Country.mmdb]
[2022-04-16T19:50:16,443][INFO ][o.e.i.g.DatabaseNodeService] [hyperion] successfully reloaded changed geoip database file [/var/folders/p9/r3gjl2912ybd827msztppr_m0000gn/T/elasticsearch-15885661857466451462/geoip-databases/C7v7WTzGQ7CnHKoZ8sbmAw/GeoLite2-Country.mmdb]
[2022-04-16T19:50:16,711][INFO ][o.e.i.g.DatabaseNodeService] [hyperion] successfully reloaded changed geoip database file [/var/folders/p9/r3gjl2912ybd827msztppr_m0000gn/T/elasticsearch-15885661857466451462/geoip-databases/C7v7WTzGQ7CnHKoZ8sbmAw/GeoLite2-City.mmdb]

and now we can see the geoip index etc and the stats about the download updates...

hyperion:elasticsearch-7.16.2 sbrown$ curl http://127.0.0.1:9200/_cat/indices/.geo*?v
health status index            uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   .geoip_databases IJM4aKLmTB2YlxEfWfwkMA   1   0         40            0     37.9mb         37.9mb
hyperion:elasticsearch-7.16.2 sbrown$ curl localhost:9200/_ingest/geoip/stats
{"stats":{"successful_downloads":3,"failed_downloads":0,"total_download_time":7733,"databases_count":3,"skipped_updates":0,"expired_databases":0},"nodes":{"C7v7WTzGQ7CnHKoZ8sbmAw":{"databases":[{"name":"GeoLite2-City.mmdb"},{"name":"GeoLite2-Country.mmdb"},{"name":"GeoLite2-ASN.mmdb"}],"files_in_temp":["GeoLite2-ASN.mmdb_elastic-geoip-database-service-agreement-LICENSE.txt","GeoLite2-ASN.mmdb_LICENSE.txt","GeoLite2-City.mmdb_LICENSE.txt","GeoLite2-Country.mmdb_elastic-geoip-database-service-agreement-LICENSE.txt","GeoLite2-ASN.mmdb","GeoLite2-City.mmdb_COPYRIGHT.txt","GeoLite2-City.mmdb","GeoLite2-City.mmdb_elastic-geoip-database-service-agreement-LICENSE.txt","GeoLite2-Country.mmdb_LICENSE.txt","GeoLite2-Country.mmdb","GeoLite2-ASN.mmdb_COPYRIGHT.txt","GeoLite2-Country.mmdb_COPYRIGHT.txt","GeoLite2-City.mmdb_README.txt"]}}}hyperion:elasticsearch-7.16.2

Now disable again
ingest.geoip.downloader.enabled: false

now it went back to the initialized the default database (the one that comes shipped) BUT does not actually load the databases into elasticsearch.

[2022-04-16T19:52:56,306][INFO ][o.e.i.g.ConfigDatabases  ] [hyperion] initialized default databases [[GeoLite2-Country.mmdb, GeoLite2-City.mmdb, GeoLite2-ASN.mmdb]], config databases [[]] and watching [/Users/sbrown/workspace/elastic-install/7.16.2/tmp/elasticsearch-7.16.2/config/ingest-geoip] for changes
[2022-04-16T19:52:56,307][INFO ][o.e.i.g.DatabaseNodeService] [hyperion] initialized database registry, using geoip-databases directory [/var/folders/p9/r3gjl2912ybd827msztppr_m0000gn/T/elasticsearch-1698940183761466503/geoip-databases/C7v7WTzGQ7CnHKoZ8sbmAw]
....
[2022-04-16T19:52:59,909][INFO ][o.e.c.m.MetadataDeleteIndexService] [hyperion] [.geoip_databases/IJM4aKLmTB2YlxEfWfwkMA] deleting index

And now there is no .geoip index (i.e. it was cleaned up)

hyperion:elasticsearch-7.16.2 sbrown$ curl http://127.0.0.1:9200/_cat/indices/.geo*?v
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size

and it not longer recognizes the downloaded databases

hyperion:elasticsearch-7.16.2 sbrown$ curl localhost:9200/_ingest/geoip/stats
{"stats":{"successful_downloads":0,"failed_downloads":0,"total_download_time":0,"databases_count":0,"skipped_updates":0,"expired_databases":0},"nodes":{}}hyperion:elasticsearch-7.16.2 

Now I do still see the "Downloaded" temp database but it is not longer used...

I would need to get clarification of what the docs mean but after you set to false no new updates will come, and the geoip databases are not actually loaded into elasticsearch.

If false , Elasticsearch does not download updates and deletes all downloaded databases.

So I think there are 2 things ... the message about initializing the default (shipped) database is a bit confusing even though it is not Actually loaded into Elasticsearch

and the fact the the temp downloads are not cleaned up.. perhaps they should be... but when set to false the geoip database are not loaded into .. nor updated into elasticsearch.