My geoip processor doesn't want to download its data?
ingest.geoip.downloader.enabled is true by default, otherwise I wouldn't be able to query the following stats, right? I have tried enabling it manually as persistent setting too, but that doesn't make a difference.
What do your elasticsearch logs look like, usually there is a line on whether it needed to download. Perhaps elasticsearch can not write / download to the temp directory... Look at the logs.
I have just upgraded to 8.0.0 on a Debian 10 system.
I am seeing a lot of this request accesses aliases with names reserved for system indices and this request accesses aliases with names reserved for system indices, but no download entries in the current logs.
There is an older entry where it initialized the database registry using a folder in /tmp/elasticsearch* but as far as I can see there isn't a similar folder at this moment.
[2022-02-10T09:12:05,471][INFO ][o.e.i.g.DatabaseNodeService] [elk-elasticsearch-01] initialized database registry, using geoip-databases directory [/tmp/elasticsearch-68890245248798155/geoip-databases/72eUP9qXTgiiQDFb8HmP1g]
I have just restarted one node and the ingest-geoip is loaded, but I don't see any messages on a download.
One thing that confuses me now is that the node is started with -Djava.io.tmpdir=/tmp/elasticsearch-7743237319263154963, but there is no such directory?
I do not have any persistent or transient settings and there is no mention of geoip in elasticsearch.yml, so it should use the default which is true:
Ok, I have now done dist-upgrades to Debian 11 on my nodes to be up to date with the whole system and I see a tmp-folder like /tmp/systemd-private-*-elasticsearch.service-*/tmp/elasticsearch-*/geoip-databases/*/ but it is empty.
In the logs I see the geoip module being loaded, but no other information on failed downloads or similar.
I am confused.. exactly what is your concern... Why do you think you have failed download.
The distribution comes with the latest geoip database so it does not need to download it initially.
It will wait 3 days to check to see if it needs a new download. Not necessarily download a new one every 3 days. MaxMind release is a new database about every 30 days is my understanding
It will / should download a new one when needed.
What is the output now of
curl -k -X GET "https://localhost:9200/_ingest/geoip/stats?pretty"
If you want to force a download you could set
ingest.geoip.downloader.enabled: false
Then restart all the nodes that will clear out the geoip database.
Then set to true and restart that will force a download of the database
My concern is missing geoip data for example for the System integration SSH login attemps dashboard. Logs have an source.ip field, but the source.geo.country_iso_code is empty.
The documents also have tags _geoip_database_unavailable_GeoLite2-City.mmdb, _geoip_database_unavailable_GeoLite2-ASN.mmdb set.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.