_geoip_lookup_failure in Logstash pipeline using GeoLite2-City.mmdb

[Carlos_T]

Hi all

When trying to enrich the following pipeline of my Logstash 8.4.3 with GeoIP info:

input {
    file {
        path => "/var/log/apache2/*.log"
        start_position => "beginning"
    }

    http {

    }
}

filter {
    fingerprint {
        source => "message"
        target => "[@metadata][fingerprint]"
        method => "MD5"
    }

    if  [log][file][path] =~ "error" {
        mutate {
            replace => { type => "error" }
        }
    } else {
        mutate {
            replace => { type => "access" }
        }

        grok {
            match => { "message" => "%{HTTPD_COMMONLOG}" }
        }

        if "_grokparsefailure" in [tags] {
            drop {}
        }

        mutate {
            convert => {
                "response" => "integer"
                "bytes" => "integer"
            }
        }

        date {
            match => [ "timestamp", "dd/MMM/yyyy:HH:mm:ss Z" ]
        }

        geoip {
            source => "source.address"
            target => "geoip"
            database => "/bdgeo/GeoLite2-City.mmdb" # Reemplaza con la ruta correcta a la base de datos de geolocalización
        }


    }

}


output {
    stdout {
        codec => rubydebug
    }

    file {
        path => "%{type}_%{+yyyy_MM_dd}.log"
    }
}

Then I run the pipeline without error and next I add a line like this to my /var/log/apache2/access.log

112.85.231.147 - - [18/May/2023:12:26:02 +0200] "GET /miweb.html HTTP/1.1" 200 543 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/113.0"

And this is the output I get:

{
         "event" => {
        "original" => "112.85.231.147 - - [18/May/2023:12:26:02 +0200] \"GET /miweb.html HTTP/1.1\" 200 543 \"-\" \"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/113.0\""
    },
    "@timestamp" => 2023-05-18T10:26:02.000Z,
      "@version" => "1",
       "message" => "112.85.231.147 - - [18/May/2023:12:26:02 +0200] \"GET /miweb.html HTTP/1.1\" 200 543 \"-\" \"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/113.0\"",
          "host" => {
        "name" => "ubuntuelk02"
    },
           "url" => {
        "original" => "/miweb.html"
    },
          "tags" => [
        [0] "_geoip_lookup_failure"
    ],
          "http" => {
         "version" => "1.1",
         "request" => {
            "method" => "GET"
        },
        "response" => {
                   "body" => {
                "bytes" => 543
            },
            "status_code" => 200
        }
    },
        "source" => {
        "address" => "112.85.231.147"
    },
          "type" => "access",
           "log" => {
        "file" => {
            "path" => "/var/log/apache2/access.log"
        }
    },
     "timestamp" => "18/May/2023:12:26:02 +0200"
}

I downloaded the 'GeoLite2 City ' database and I've 'chmod'ed it so the path and the file are reachable and readable. I've tried with different IP's too.

Does someone please have any idea about what could be wrong?

Thank you in advance.

[Rios]

Most likely is ECS. You should use field source.ip or destination.ip
Use: source => "[source][ip]"
Also is possible to disable ECS.

        geoip {
            source => "source.address"
            target => "geoip"
            database => "/bdgeo/GeoLite2-City.mmdb" 
            ecs_compatibility => "disabled"
        }

[leandrojmp (Leandro Pereira)]

@Rios

source => "source.address"

Also, this needs to be [source][address]

[Rios]

Good point, thx.

[Carlos_T]

It was a combination of what you both said:

geoip {
            source => "[source][address]"
            target => "geoip"
            database => "/bdgeo/GeoLite2-City.mmdb" # Reemplaza con la ruta correcta a la base de datos de geolocalización
            ecs_compatibility => "disabled"
        }

By using this pipeline

Thank you very much Rios and Leandro.

That was fast an efficient