GeoIP Filter in ECS-Compatiblity mode requires a `target` when `source` is not an `ip` sub-field, eg. [client][ip]

Hi,

i'm trying to setup
Parsing Logs with Logstash | Logstash Reference [8.11] | Elastic.
But i get errors:

[2024-01-04T00:06:45,246][ERROR][logstash.javapipeline    ][main] Pipeline error {:pipeline_id=>"main", :exception=>#<LogStash::ConfigurationError: GeoIP Filter in ECS-Compatiblity
mode requires a `target` when `source` is not an `ip` sub-field, eg. [client][ip]>

My pipeline looks like this:

  beats {
    ssl => "false"
    port => "5044"
  }
}

filter {
    grok {
        match => { "message" => "%{COMBINEDAPACHELOG}"}
    }
    geoip {
        source => "clientip"
    }
}

output {
  stdout { codec => rubydebug }
}

When i comment out the geoip filter i get this:

{
         "input" => {
        "type" => "log"
    },
           "ecs" => {
        "version" => "8.0.0"
    },
      "@version" => "1",
           "log" => {
        "offset" => 34227,
          "file" => {
            "path" => "/root/logstash-tutorial.log"
        }
    },
          "host" => {
        "name" => "suricata"
    },
         "agent" => {
        "ephemeral_id" => "601bd8bf-5443-410f-8712-56e85a58295e",
                "name" => "suricata",
             "version" => "8.11.3",
                  "id" => "d64a8bc7-2c9f-4ac7-a499-7daa78227552",
                "type" => "filebeat"
    },
        "source" => {
        "address" => "86.1.76.62"
    },
     "timestamp" => "04/Jan/2015:05:30:37 +0000",
    "@timestamp" => 2024-01-03T19:51:34.436Z,
          "http" => {
        "response" => {
            "status_code" => 200,
                   "body" => {
                "bytes" => 4877
            }
        },
         "version" => "1.1",
         "request" => {
            "referrer" => "http://www.semicomplete.com/projects/xdotool/",
              "method" => "GET"
        }
    },
           "url" => {
        "original" => "/style2.css"
    },
          "tags" => [
        [0] "beats_input_codec_plain_applied"
    ],
    "user_agent" => {
        "original" => "Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20140205 Firefox/24.0 Iceweasel/24.3.0"
    },
       "message" => "86.1.76.62 - - [04/Jan/2015:05:30:37 +0000] \"GET /style2.css HTTP/1.1\" 200 4877 \"http://www.semicomplete.com/projects/xdotool/\" \"Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20140205 Firefox/24.0 Iceweasel/24.3.0\"",
         "event" => {
        "original" => "86.1.76.62 - - [04/Jan/2015:05:30:37 +0000] \"GET /style2.css HTTP/1.1\" 200 4877 \"http://www.semicomplete.com/projects/xdotool/\" \"Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20140205 Firefox/24.0 Iceweasel/24.3.0\""
    }
}

I don't see "geoip". Is this the problem?
I played around with the filter and tried:

filter {
    grok {
        match => { "message" => "%{COMBINEDAPACHELOG}"}
    }
    geoip {
        source => "[source][address]"
    }
}

but didn't suceed.

Bernd

If your geoip filter has source that looks like [someField][ip], then it assumes that it is OK to add all the geoip fields under [someField]. Otherwise it needs you to tell it where to add those fields. You will also need an index template if you want to end up with a geo_point in an ES index.

If ECS is disabled then the target will be [geoip], which limits you to one geoip filter, which is not good, since a lot of people are dealing with logs that have variable sources and destinations.

1 Like

Hi,
this is now my pipeline:

    grok {
         match => { "message" => "%{COMBINEDAPACHELOG}"}
     }
      geoip {
         source => "[source][address]"
         target => "source"
     }
}

I get the following output:

        "address" => "86.1.76.62",
            "geo" => {
            "country_iso_code" => "GB",
                    "timezone" => "Europe/London",
                "country_name" => "United Kingdom",
                   "city_name" => "Burnley",
                 "region_name" => "Lancashire",
                 "postal_code" => "BB10",
             "region_iso_code" => "GB-LAN",
                    "location" => {
                "lon" => -2.2372,
                "lat" => 53.8265
            },

For me that looks fine.
What do you mean by "index template" ?

Thanks.

Bernd

They are a way to tell elasticsearch how to interpret the data that you write to it.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.