# GeoIP processor / setup?

**URL:** <https://discuss.elastic.co/t/geoip-processor-setup/298227>\
**Category:** Kibana\
**Tags:** ingest-pipeline\
**Created:** [February 24, 2022, 11:21pm UTC](https://discuss.elastic.co/t/geoip-processor-setup/298227 "2022-02-24T23:21:17Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Balu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/balu/32/37569_2.png) [@Balu](https://discuss.elastic.co/u/Balu)\
**Post date:** [February 24, 2022, 11:21pm UTC](https://discuss.elastic.co/t/geoip-processor-setup/298227/1 "2022-02-24T23:21:17Z")

</div>

My geoip processor doesn't want to download its data?

`ingest.geoip.downloader.enabled` is `true` by default, otherwise I wouldn't be able to query the following stats, right? I have tried enabling it manually as persistent setting too, but that doesn't make a difference.

```auto
$ curl -k -X GET "https://localhost:9200/_ingest/geoip/stats?pretty"
{
  "stats" : {
    "successful_downloads" : 0,
    "failed_downloads" : 0,
    "total_download_time" : 0,
    "databases_count" : 0,
    "skipped_updates" : 0,
    "expired_databases" : 0
  },
  "nodes" : { }
}

```

Do I need to enable something else? To me the stats looks like no attempts were made at all… Also I am confused why it doesn't list any nodes?

I can connect to the Elastic GeoIP endpoint `https://geoip.elastic.co/v1/database` by using `curl`, but I only get a `400: Bad Request` reply.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [February 24, 2022, 11:35pm UTC](https://discuss.elastic.co/t/geoip-processor-setup/298227/2 "2022-02-24T23:35:35Z")

</div>

What Version Stack are you on?

What do your elasticsearch logs look like, usually there is a line on whether it needed to download. Perhaps elasticsearch can not write / download to the temp directory... Look at the logs.

Also are you sure you have the setting correct

From the docs [here](https://www.elastic.co/guide/en/elasticsearch/reference/current/geoip-stats-api.html)

> ### 
> 
> Prerequisites
> 
> - If [`ingest.geoip.downloader.enabled`](https://www.elastic.co/guide/en/elasticsearch/reference/current/geoip-processor.html#ingest-geoip-downloader-enabled) **is disabled, this API returns zero values and an empty `nodes` object.**

Which what your result is.

You should see something like

```auto
curl -k -X GET "http://localhost:9200/_ingest/geoip/stats?pretty"
{
  "stats" : {
    "successful_downloads" : 3,
    "failed_downloads" : 0,
    "total_download_time" : 4823,
    "databases_count" : 3,
    "skipped_updates" : 0,
    "expired_databases" : 0
  },
  "nodes" : {
    "6OTEKpyUTTuJ8057Ucd_Tg" : {
      "databases" : [
        {
          "name" : "GeoLite2-ASN.mmdb"
        },
        {
          "name" : "GeoLite2-Country.mmdb"
        },
        {
          "name" : "GeoLite2-City.mmdb"
        }
      ],
      "files_in_temp" : [
        "GeoLite2-ASN.mmdb_elastic-geoip-database-service-agreement-LICENSE.txt",
        "GeoLite2-ASN.mmdb_LICENSE.txt",
        "GeoLite2-City.mmdb_LICENSE.txt",
        "GeoLite2-Country.mmdb_elastic-geoip-database-service-agreement-LICENSE.txt",
        "GeoLite2-ASN.mmdb",
        "GeoLite2-City.mmdb_COPYRIGHT.txt",
        "GeoLite2-City.mmdb",
        "GeoLite2-City.mmdb_elastic-geoip-database-service-agreement-LICENSE.txt",
        "GeoLite2-Country.mmdb_LICENSE.txt",
        "GeoLite2-Country.mmdb",
        "GeoLite2-ASN.mmdb_COPYRIGHT.txt",
        "GeoLite2-Country.mmdb_COPYRIGHT.txt",
        "GeoLite2-City.mmdb_README.txt"
      ]
    }
  }
}

```

---

<div class="post-metadata">

**Author:** ![Balu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/balu/32/37569_2.png) [@Balu](https://discuss.elastic.co/u/Balu)\
**Post date:** [February 25, 2022, 7:33am UTC](https://discuss.elastic.co/t/geoip-processor-setup/298227/3 "2022-02-25T07:33:40Z")

</div>

I have just upgraded to 8.0.0 on a Debian 10 system.

I am seeing a lot of `this request accesses aliases with names reserved for system indices` and `this request accesses aliases with names reserved for system indices`, but no download entries in the current logs.

There is an older entry where it initialized the database registry using a folder in `/tmp/elasticsearch*` but as far as I can see there isn't a similar folder at this moment.

```auto
[2022-02-10T09:12:05,471][INFO][o.e.i.g.DatabaseNodeService] [elk-elasticsearch-01] initialized database registry, using geoip-databases directory [/tmp/elasticsearch-68890245248798155/geoip-databases/72eUP9qXTgiiQDFb8HmP1g]

```

I have just restarted one node and the `ingest-geoip` is loaded, but I don't see any messages on a download.

One thing that confuses me now is that the node is started with `-Djava.io.tmpdir=/tmp/elasticsearch-7743237319263154963`, but there is no such directory?

I do not have any persistent or transient settings and there is no mention of geoip in `elasticsearch.yml`, so it should use the default which is `true`:

```auto
$ curl -k -X GET "https://localhost:9200/_cluster/settings?pretty"
{
  "persistent" : { },
  "transient" : { }
}

```

---

<div class="post-metadata">

**Author:** ![Balu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/balu/32/37569_2.png) [@Balu](https://discuss.elastic.co/u/Balu)\
**Post date:** [February 25, 2022, 10:17am UTC](https://discuss.elastic.co/t/geoip-processor-setup/298227/4 "2022-02-25T10:17:15Z")

</div>

Ok, I have now done dist-upgrades to Debian 11 on my nodes to be up to date with the whole system and I see a `tmp`-folder like `/tmp/systemd-private-*-elasticsearch.service-*/tmp/elasticsearch-*/geoip-databases/*/` but it is empty.

In the logs I see the geoip module being loaded, but no other information on failed downloads or similar.

```auto
# rg -i geoip /var/log/elasticsearch/elk-elasticsearch-prod.log
242:[2022-02-25T09:13:11,971][INFO][o.e.p.PluginsService] [elk-elasticsearch-01] loaded module [ingest-geoip]

```

Is it possible it is now waiting for the default three days for the next download? If so - is there a way to trigger the download manually?

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [February 25, 2022, 3:03pm UTC](https://discuss.elastic.co/t/geoip-processor-setup/298227/5 "2022-02-25T15:03:57Z")

</div>

I am confused.. exactly what is your concern... Why do you think you have failed download.

The distribution comes with the latest geoip database so it does not need to download it initially.

It will wait 3 days to check to see if it needs a new download. Not necessarily download a new one every 3 days. MaxMind release is a new database about every 30 days is my understanding

It will / should download a new one when needed.

What is the output now of

`curl -k -X GET "https://localhost:9200/_ingest/geoip/stats?pretty"`

If you want to force a download you could set

`ingest.geoip.downloader.enabled: false`  
Then restart all the nodes that will clear out the geoip database.

Then set to `true` and restart that will force a download of the database

---

<div class="post-metadata">

**Author:** ![Balu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/balu/32/37569_2.png) [@Balu](https://discuss.elastic.co/u/Balu)\
**Post date:** [February 25, 2022, 3:30pm UTC](https://discuss.elastic.co/t/geoip-processor-setup/298227/6 "2022-02-25T15:30:49Z")

</div>

I am confused myself 🙂

My concern is missing geoip data for example for the `System` integration `SSH login attemps` dashboard. Logs have an `source.ip` field, but the `source.geo.country_iso_code` is empty.

The documents also have tags `_geoip_database_unavailable_GeoLite2-City.mmdb, _geoip_database_unavailable_GeoLite2-ASN.mmdb` set.

```auto
curl -k -X GET "https://localhost:9200/_ingest/geoip/stats?pretty"
{
  "stats" : {
    "successful_downloads" : 0,
    "failed_downloads" : 0,
    "total_download_time" : 0,
    "databases_count" : 0,
    "skipped_updates" : 0,
    "expired_databases" : 0
  },
  "nodes" : { }
}

```

I will try to "turn it off and on again" and report back…

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [February 25, 2022, 4:01pm UTC](https://discuss.elastic.co/t/geoip-processor-setup/298227/7 "2022-02-25T16:01:12Z")

</div>

Intersesting... OK now I understand your issue.... Yes the DBs appear to be missing and that is why your geoip is not working...

turn off / on and looks at the logs carefully

When you turn back on you should see logs something like (These are from 7.17.0) I have not tested on 8.0 yet

```auto
[2022-02-25T08:00:09,535][INFO][o.e.i.g.DatabaseNodeService] [hyperion] downloading geoip database [GeoLite2-ASN.mmdb] to [/var/folders/p9/r3gjl2912ybd827msztppr_m0000gn/T/elasticsearch-8287626989138835694/geoip-databases/0gvfvFL5QP6-SJ1MzcQygA/GeoLite2-ASN.mmdb.tmp.gz]
[2022-02-25T08:00:09,549][INFO][o.e.i.g.GeoIpDownloader] [hyperion] updated geoip database [GeoLite2-ASN.mmdb]
[2022-02-25T08:00:09,564][INFO][o.e.i.g.GeoIpDownloader] [hyperion] updating geoip database [GeoLite2-City.mmdb]
[2022-02-25T08:00:09,704][INFO][o.e.i.g.DatabaseNodeService] [hyperion] successfully reloaded changed geoip database file [/var/folders/p9/r3gjl2912ybd827msztppr_m0000gn/T/elasticsearch-8287626989138835694/geoip-databases/0gvfvFL5QP6-SJ1MzcQygA/GeoLite2-ASN.mmdb]
[2022-02-25T08:00:13,908][INFO][o.e.i.g.DatabaseNodeService] [hyperion] downloading geoip database [GeoLite2-City.mmdb] to [/var/folders/p9/r3gjl2912ybd827msztppr_m0000gn/T/elasticsearch-8287626989138835694/geoip-databases/0gvfvFL5QP6-SJ1MzcQygA/GeoLite2-City.mmdb.tmp.gz]
[2022-02-25T08:00:13,922][INFO][o.e.i.g.GeoIpDownloader] [hyperion] updated geoip database [GeoLite2-City.mmdb]
[2022-02-25T08:00:13,923][INFO][o.e.i.g.GeoIpDownloader] [hyperion] updating geoip database [GeoLite2-Country.mmdb]
[2022-02-25T08:00:15,316][INFO][o.e.i.g.DatabaseNodeService] [hyperion] downloading geoip database [GeoLite2-Country.mmdb] to [/var/folders/p9/r3gjl2912ybd827msztppr_m0000gn/T/elasticsearch-8287626989138835694/geoip-databases/0gvfvFL5QP6-SJ1MzcQygA/GeoLite2-Country.mmdb.tmp.gz]
[2022-02-25T08:00:15,331][INFO][o.e.i.g.GeoIpDownloader] [hyperion] updated geoip database [GeoLite2-Country.mmdb]
[2022-02-25T08:00:15,389][INFO][o.e.i.g.DatabaseNodeService] [hyperion] successfully reloaded changed geoip database file [/var/folders/p9/r3gjl2912ybd827msztppr_m0000gn/T/elasticsearch-8287626989138835694/geoip-databases/0gvfvFL5QP6-SJ1MzcQygA/GeoLite2-Country.mmdb]
[2022-02-25T08:00:15,705][INFO][o.e.i.g.DatabaseNodeService] [hyperion] successfully reloaded changed geoip database file [/var/folders/p9/r3gjl2912ybd827msztppr_m0000gn/T/elasticsearch-8287626989138835694/geoip-databases/0gvfvFL5QP6-SJ1MzcQygA/GeoLite2-City.mmdb]

```

---

<div class="post-metadata">

**Author:** ![Balu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/balu/32/37569_2.png) [@Balu](https://discuss.elastic.co/u/Balu)\
**Post date:** [February 25, 2022, 5:31pm UTC](https://discuss.elastic.co/t/geoip-processor-setup/298227/8 "2022-02-25T17:31:07Z")

</div>

I guess, the usual "turning it off and on again" fix worked as it always does...

I disabled the geoip processor:

```auto
curl -k -X PUT "https://localhost:9200/_cluster/settings?pretty" -H 'Content-Type: application/json' -d'
{
  "persistent" : {
    "ingest.geoip.downloader.enabled" : false
  }
}
'

```

Then stopped and started each node, waiting for the cluster to recover every time.

Then I set `ingest.geoip.downloader.enabled` to `null` (making it default to `true`) and the system logged:

```auto
[2022-02-25T18:24:39,462][INFO][o.e.c.s.ClusterSettings] [node-01] updating [ingest.geoip.downloader.enabled] from [false] to [true] │
[2022-02-25T18:24:41,693][INFO][o.e.i.g.DatabaseNodeService] [node-01] successfully loaded geoip database file [GeoLite2-ASN.mmdb] │
[2022-02-25T18:24:44,967][INFO][o.e.i.g.DatabaseNodeService] [node-01] successfully loaded geoip database file [GeoLite2-Country.mmdb] │
[2022-02-25T18:24:45,052][INFO][o.e.i.g.DatabaseNodeService] [node-01] successfully loaded geoip database file [GeoLite2-City.mmdb] │

```

Thanks for your help, Stephen

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [February 25, 2022, 6:27pm UTC](https://discuss.elastic.co/t/geoip-processor-setup/298227/9 "2022-02-25T18:27:24Z")

</div>

Restart / Reboot / Reinstall ... 3 Truths of IT 🙂  
Glad it is working for you ...

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 25, 2022, 6:27pm UTC](https://discuss.elastic.co/t/geoip-processor-setup/298227/10 "2022-03-25T18:27:40Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.