Packetbeat 7.1 and Geoip setup

Elastic Stack Beats
21

From looking at the 2 files can you determine What I have wrong?

22

Nope, there doesn't seem to be any geoip info in there which is unusual.

23

What is the next step?

24

I'll see if I can get someone a bit more knowledgable involved :slight_smile:

25

@Marcell0e

The source and destination IPs in the document you provided belong to private networks, thus the geoip processor using the default database cannot add geolocation for them.

Can you check with public internet IPs?

1 Like
26

Is there away to map these IPs to a location of my choosing? Every post i have read they are using Logstash. I'm not using Logstash. I'm still trying to figure out Elasticsearch.

27

I think the process would be similar as with Logstash. Create your custom mmdb database, put it in the directory that Elasticsearch expects it (elasticsearch/modules/geo-ip).

Here's some article on how to create an mmdb:
https://blog.maxmind.com/2015/09/29/building-your-own-mmdb-database-for-fun-and-profit/

28

Will Elasticsearch be able to use both mmdb database to find IPs or just 1?

29

One Geoip processor instance can only read from a single database. That gives you two options:

  • Create a new database that has your custom IPs plus the contents of the default database (I don't know how to do it or if it's possible at all, as I never worked with MMDB files myself).
  • In your ingest pipeline, have 2 geoip processors for every field you want populated (source.geo, destination.geo). The second one contains an if condition so that it executes only if the first one didn't populate the target field ("if": "ctx?.source?.geo == null").
30

How would I go about setting up your second suggestion?

31

You have your original batch of processors:

{
      "geoip": {
        "field": "client.ip",
        "target_field": "client.geo",
        "ignore_missing": true
      }
    },
    {
      "geoip": {
        "field": "source.ip",
        "target_field": "source.geo",
        "ignore_missing": true
      }
    },
    {
      "geoip": {
        "field": "destination.ip",
        "target_field": "destination.geo",
        "ignore_missing": true
      }
    },
    {
      "geoip": {
        "field": "server.ip",
        "target_field": "server.geo",
        "ignore_missing": true
      }
    },
    {
      "geoip": {
        "field": "host.ip",
        "target_field": "host.geo",
        "ignore_missing": true
      }
    },

and them the same with your custom database and a condition to populate only missing fields:

{
      "geoip": {
        "field": "client.ip",
        "target_field": "client.geo",
        "ignore_missing": true,
        "database_file": "my_custom.mmdb",
        "if": "ctx?.client?.geo == null"
      }
    },
    {
      "geoip": {
        "field": "source.ip",
        "target_field": "source.geo",
        "ignore_missing": true,
        "database_file": "my_custom.mmdb",
        "if": "ctx?.source?.geo == null"
      }
    },
    {
      "geoip": {
        "field": "destination.ip",
        "target_field": "destination.geo",
        "ignore_missing": true,
        "database_file": "my_custom.mmdb",
        "if": "ctx?.destination?.geo == null"
      }
    },
    {
      "geoip": {
        "field": "server.ip",
        "target_field": "server.geo",
        "ignore_missing": true,
        "database_file": "my_custom.mmdb",
        "if": "ctx?.server?.geo == null"
      }
    },
    {
      "geoip": {
        "field": "host.ip",
        "target_field": "host.geo",
        "ignore_missing": true,
        "database_file": "my_custom.mmdb",
        "if": "ctx?.host?.geo == null"
      }
    }

Warning: untested

32

Would this be easier than creating a new MMDb?

Creating geoip data for internal networks

Is this possible?

33

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.