GeoIP problems

Elastic Stack Elasticsearch
1

Hello, I make a post because I'm not be able to set up GeoIP fields and map in my Elasticsearch.

My deployment is a VM with Elastic+Kibana+Elastic-Agent (standalone) On Premise without license and with Internet Connection. VM uses Linux SUSE.

I have two integrations (PaloAlto Firewall and FortiGate Firewall through Elastic-Agent).

I connect to Kibana from a browser on a virtual desktop that does not have access to the Internet, but the virtual machine where Elasticsearch is hosted does have access to the Internet.

The problem is that the Kibana map does not load. And the Fortigate and PaloAlto logs also appear with the following labels:
_geoip_database_unavailable_GeoLite2-City.mmdb, _geoip_database_unavailable_GeoLite2-City.mmdb, _geoip_database_unavailable_GeoLite2-ASN.mmdb, _geoip_database_unavailable_GeoLite2-ASN.mmdb

How could I solve this and be able to create a map with the IPs?

Thxs

2

Hi @akrog79

What version are you on?

Did you make any changes in the elasticsearch.yml with respect to geoip settings?

Are you sure your elasticsearch VM has access to the internet?

From the Elasticsearch VM can you curl the following

curl https://geoip.elastic.co/v1/database?elastic_geoip_service_tos=agree

Did you review this

Can you go to Kibana - Dev Tools and run the 2 following commands and provide the output, please show the command and the output.

GET /

GET _ingest/geoip/stats

Then run

PUT _ingest/pipeline/geoip
{
  "description" : "Add geoip info",
  "processors" : [
    {
      "geoip" : {
        "field" : "ip"
      }
    }
  ]
}

# Wait about 1 minute

GET _ingest/geoip/stats

PUT my-index-000001/_doc/my_id?pipeline=geoip
{
  "ip": "89.160.20.128"
}
GET my-index-000001/_doc/my_id

Let me know what you see...

1 Like
3

From Elastic Search to Elasticsearch

4

Thanks @stephenb for your message!!

v8.14.1 with x-pack enabled.

Yes, my VM has acces to Internet

esearch:~ # curl https://geoip.elastic.co/v1/database?elastic_geoip_service_tos=agree
[{"age":56668,"md5_hash":"4c175b693b0fda333092353ab0489014","name":"GeoLite2-ASN.tgz","provider":"maxmind","updated":1728000027,"url":"https://storage.googleapis.com/elastic-paisano-production/maxmind/GeoLite2-ASN.tgz?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=elastic-paisano-production%40elastic-apps-163815.iam.gserviceaccount.com%2F20241004%2Fhenk%2Fstorage%2Fgoog4_request&X-Goog-Date=20241004T000027Z&X-Goog-Expires=86400&X-Goog-SignedHeaders=host&X-Goog-Signature=1949ab87811b7035653a2a9df552c1becd42a010adfc14cbe9d87882f96dfdc2279f9b69f9c910b5132d1e981000b32f5914ab8d8d5f62f70b183de21793a076e0c2af76a8696fd4ace05add551c4247d6a341b1eae360e2824202817b1d50a834dbabbe12e6ce1e4f97160e94afc3a28f0af807992d1844c1504c719ecf7cb66bc89468d8c9d09cf32454fc4569b00b561be410f280f1708bc06b37b68c104f0466fb451611bc47aedabd6d7d321a1308aec952ae434bb6be3e128b5f164d2d362a3248bec37502caf6c8b21a87ad799345dfe2d5d585fa8fe53feb9c204046237472496663ee22b466755b8500e7bd23e84f808622fcec781cb992369091f8"},{"age":190076,"md5_hash":"ff12447a40c3830e42fd417f49ea6386","name":"GeoLite2-City.tgz","provider":"maxmind","updated":1728000027,"url":"https://storage.googleapis.com/elastic-paisano-production/maxmind/GeoLite2-City.tgz?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=elastic-paisano-production%40elastic-apps-163815.iam.gserviceaccount.com%2F20241004%2Fhenk%2Fstorage%2Fgoog4_request&X-Goog-Date=20241004T000027Z&X-Goog-Expires=86400&X-Goog-SignedHeaders=host&X-Goog-Signature=a3349cdd7b7c59649ca3c6398a666e779e5c83823de6bf769683b671ed2bc3fa8f6394c36ad812677b549252cab862ed7f6ca5ca5839d8d3ce24ae89aef82ae127b32a592319bb0eefb7d4ff6e34636f6fb400cfdfa491fc40de9e9ac48887e3cc99f66e52a707ea3bc162027ca706d21d339360c5c78fb2b026b02e211b32b0df4c59b6ac199e0764107123999850de8247db1ae1ac9324bba32d2c845f4c9a4e37430ce67978aad4315264a9be713c35aa6fa27f3f3052b0d5f340896ad83239e33f253dff8c172a4ba4fbdaae896ce8b4c4b88acb69c2b4f89d51df8af1b01dc5d5d6e595833d75ce1a18d60d11bc9f8b21a5d6fee327de1d4fbc6d1e3a3f"},{"age":189762,"md5_hash":"b9c472752d960b50f952c8469368188c","name":"GeoLite2-Country.tgz","provider":"maxmind","updated":1728000027,"url":"https://storage.googleapis.com/elastic-paisano-production/maxmind/GeoLite2-Country.tgz?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=elastic-paisano-production%40elastic-apps-163815.iam.gserviceaccount.com%2F20241004%2Fhenk%2Fstorage%2Fgoog4_request&X-Goog-Date=20241004T000027Z&X-Goog-Expires=86400&X-Goog-SignedHeaders=host&X-Goog-Signature=47657521363eae5c09a1016f6ad28ea537b5bf34800ac518fda723db778a3a4cb299bddf4e6c2d2f42384b9ef9dd160a660326b65cec79f89998e69004d4148c516dbb92ff70e1a5861f9d0be47aa7d3ebfb7a07b92d66da11276d4a78daa2db29e8c8f7f72c5313b0234d78eebdb3080bca3d21713ca6e157ff7dcf9af0fef8109b11daf90b04bc5d3043d95f632d0e32fa0cac4f530c198f4cd474d0cdcacfca25264ac31cdb55243deed649c117371b1c7460b0d232c6cb42b2e52bfcded0acef29dda642ff8bec4a01ccb42b451ff95f7ef412831003bc719056bca1fe512fec98cbfc95f52243ca6186bacf7f39849771017de85c453a1f3e907cdbdf74"}]

I put this * ingest.geoip.downloader.eager.download: true on elasticsearch.yml and restart service.

Output GET /

{
  "name": "node-1",
  "cluster_name": "elasticsearch",
  "cluster_uuid": "ckWQLtWyTF2ofAdoaH5aiA",
  "version": {
    "number": "8.14.1",
    "build_flavor": "default",
    "build_type": "rpm",
    "build_hash": "93a57a1a76f556d8aee6a90d1a95b06187501310",
    "build_date": "2024-06-10T23:35:17.114581191Z",
    "build_snapshot": false,
    "lucene_version": "9.10.0",
    "minimum_wire_compatibility_version": "7.17.0",
    "minimum_index_compatibility_version": "7.0.0"
  },
  "tagline": "You Know, for Search"
}

Output GET _ingest/geoip/stats

{
  "stats": {
    "successful_downloads": 0,
    "failed_downloads": 1,
    "total_download_time": 0,
    "databases_count": 0,
    "skipped_updates": 0,
    "expired_databases": 3
  },
  "nodes": {}
}

INPUT PUT _ingest/pipeline/geoip

OUTPUT
{
  "acknowledged": true
}

INPUT GET _ingest/geoip/stats

OUTPUT
{
  "stats": {
    "successful_downloads": 0,
    "failed_downloads": 1,
    "total_download_time": 0,
    "databases_count": 0,
    "skipped_updates": 0,
    "expired_databases": 3
  },
  "nodes": {}
}

INPUT PUT my-index-000001/_doc/my_id?pipeline=geoip

OUTPUT
{
  "_index": "my-index-000001",
  "_id": "my_id",
  "_version": 1,
  "result": "created",
  "_shards": {
    "total": 2,
    "successful": 1,
    "failed": 0
  },
  "_seq_no": 0,
  "_primary_term": 1
}

INPUT GET my-index-000001/_doc/my_id

OUTPUT
{
  "_index": "my-index-000001",
  "_id": "my_id",
  "_version": 1,
  "_seq_no": 0,
  "_primary_term": 1,
  "found": true,
  "_source": {
    "ip": "89.160.20.128",
    "tags": [
      "_geoip_database_unavailable_GeoLite2-City.mmdb"
    ]
  }
}
5

So, the actual downloads come from buckets on GCP. Perhaps your firewall is blocking them...

Go to your elasticsearch VM and try

curl -O "https://storage.googleapis.com/elastic-paisano-production/maxmind/GeoLite2-City.tgz?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=elastic-paisano-production%40elastic-apps-163815.iam.gserviceaccount.com%2F20241004%2Fhenk%2Fstorage%2Fgoog4_request&X-Goog-Date=20241004T000027Z&X-Goog-Expires=86400&X-Goog-SignedHeaders=host&X-Goog-Signature=a3349cdd7b7c59649ca3c6398a666e779e5c83823de6bf769683b671ed2bc3fa8f6394c36ad812677b549252cab862ed7f6ca5ca5839d8d3ce24ae89aef82ae127b32a592319bb0eefb7d4ff6e34636f6fb400cfdfa491fc40de9e9ac48887e3cc99f66e52a707ea3bc162027ca706d21d339360c5c78fb2b026b02e211b32b0df4c59b6ac199e0764107123999850de8247db1ae1ac9324bba32d2c845f4c9a4e37430ce67978aad4315264a9be713c35aa6fa27f3f3052b0d5f340896ad83239e33f253dff8c172a4ba4fbdaae896ce8b4c4b88acb69c2b4f89d51df8af1b01dc5d5d6e595833d75ce1a18d60d11bc9f8b21a5d6fee327de1d4fbc6d1e3a3f"

You should get a tar.gz...

If not you may need to manage manually...

1 Like
6

That's right, I was having a UTM block that I didn't see. Now the .tar.gz is downloaded with curl but the map still doesn't appear in Analytics --> Maps and I still get the failed_download error if I run the previous query.

I trying to restart elastic service but doesn't download automaticaly

7

set this to false

ingest.geoip.downloader.enabled: false

Restart elasticsearch let it come all the way up...

then reset to true
ingest.geoip.downloader.enabled: true

Then restart...let it come all the way up...

Then look at the stats...

There is also a tmp directory involved which must be writable / have space....
You can look at the elasticsearch logs

Elasticsearch stores downloaded database files in each node’s temporary directory at $ES_TMPDIR/geoip-databases/<node_id> .

1 Like
8 
{
  "_index": "my-index-000001",
  "_id": "my_id",
  "_version": 2,
  "_seq_no": 1,
  "_primary_term": 7,
  "found": true,
  "_source": {
    "ip": "89.160.20.128",
    "geoip": {
      "continent_name": "Europe",
      "country_iso_code": "SE",
      "country_name": "Sweden",
      "location": {
        "lon": 18.056,
        "lat": 59.3247
      }
    }
  }
}

Success!! Nice!!

But I still can't see the map. When I access Analytics --> Maps the basemap appears red with an X and says "An error occurred when loading layer data".

Remember that to view kibana I use a browser on a virtual desktop that does not have access to the Internet.

If I inspect the page in Console, the following events appear:

telemetry.elastic.co/v3/send/kibana-browser:1
Failed to load resource: net::ERR_CONNECTION_TIMED_OUT

Unable to access fonts from Elastic Maps Service (EMS). To avoid unnecessary EMS requests, set 'map.includeElasticMapsService: false' in 'kibana.yml'. For more details please visit: Connect to Elastic Maps Service | Kibana Guide [8.14] | Elastic

9

Now I would open I specific topic on that with a specific Title and specific details ... Not my area of expertise... and those that do... probably won't read this as the title is nothing to do with Maps..

But yeah pretty sure you need internet access that is where all the base maps come from