Cannot create map layer with GEOIP with KIBANA 8.12.1

Elastic Stack Kibana
1

Hello,

I´m trying to create a map from GEOIP coordinate retrieved from a log file.
My workflow goes this way :
logfile -> filebeat -> logstash -> es

That workflow worked as expected with ELK stack version 7
In version 8, when I try to select the dataview that contains my data, it fails with :

Data view does not contain any geospatial fields

Whereas, I can see these fields in the index for instance :

geoip.city_name
	
Toulouse
geoip.continent_code
	
EU
geoip.country_code2
	
FR
geoip.country_code3
	
FR
geoip.country_name
	
France
geoip.ip
	
92.88.xx.xx
geoip.latitude
	
43.605
geoip.location.lat
	
43.605
geoip.location.lon
	
1.445
geoip.longitude
	
1.445
geoip.postal_code
	
31000
geoip.region_code
	
31
geoip.region_name
	
Upper Garonne
geoip.timezone
	
Europe/Paris

What did I do wrong ?

2

The data format for the geospatial data has been changed to geopoint Geopoint field type | Elasticsearch Guide [8.12] | Elastic in 8.x so you either have to change the mapping and reingest the data OR you could create a geopoint field using a runtime field Map a runtime field | Elasticsearch Guide [8.12] | Elastic

3

Thanks for your answer but I don't really understand it, I'm probably doing something wrong in ELK8 that was easy in ELK7

For this workflow, I want to ingest xferlog from a FTP server that look like this :

2024/02/26 07:50:20 31.218.137.72 user1 STOR 3431594 /testfile

In my worklow, Filebeat does nothing but shipping the log (+ 1 or 2 extra fields) to Logstash that parses is this way :

       grok {
	    patterns_dir => [ "/usr/share/logstash/patterns" ]
            match => [
	      "message", "%{TIMESTAMP_XFERLOG:timestamp} (?::ffff:)?%{IP:ip} %{NOTSPACE:user} %{WORD:cmd} %{NUMBER:size} %{PATH:file}"
	    ]
       }
	geoip {
          source => "ip"
          ecs_compatibility => "disabled"
        }

Maybe I'm doing something wrong with "ecs_compatibility => "disabled"" but I did not figure it yet...
I've had to add it to make Logstash work with my settings

And logstash gives me this output :

{    "@timestamp" => 2024-02-26T07:50:20.000Z,
         "geoip" => {
         "country_code3" => "AE",
             "city_name" => "Dubai",
         "country_code2" => "AE",
           "region_code" => "DU",
        "continent_code" => "AS",
              "timezone" => "Asia/Dubai",
              "location" => {
            "lon" => 55.298,
            "lat" => 25.0731
        },
              "latitude" => 25.0731,
          "country_name" => "United Arab Emirates",
                    "ip" => "31.218.137.72",
           "region_name" => "Dubai",
             "longitude" => 55.298
    },
         "event" => {
        "original" => "2024/02/26 07:50:20 31.218.137.72 user1 STOR 3431594 /testfile"
    },
           "cmd" => "STOR",
          "size" => 3431594,
          "file" => "/testfile",
      "@version" => "1",
       "message" => "2024/02/26 07:50:20 31.218.137.72 user1 STOR 3431594 /testfile",
          "type" => "xferlog",
            "ip" => "31.218.137.72",
          "user" => "user1"
}

What mapping and how should I modify it as long as I did not have to modify anything when I was running it in ELK7 version ?

Regards.

4

that ecs_compatibility flag is the reason Logstash outputs the geoip field in the old format. By default ecs_compatibility is enabled and in that case it will output the geoip data in the new ECS style format.
That flag is for people that still use older versions of the stack to look at the data.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.