Building a Tile Map in Kibana

Elastic Stack Kibana
1

I'm very new to this. I have an log file that I parsed into

 match => [
         "message", "%{SYSLOGTIMESTAMP:timestamp} %{IPORHOST:logsource}( *)Login Success: \[%{NOTSPACE:user}\] \(%{IP:client_ip}:%{POSINT:client_port}\)]
 geoip {
         source => "client_ip"
         target => "geoip"
         add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}" ]
         add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}"  ]
 }
 mutate {
         convert => [ "[geoip][coordinates]", "float"]
 }

The client_ip field holds the IP address that I want to build a Tile Map with. I followed this blog (which I'm now thinking is out-of-date and incorrect) https://www.digitalocean.com/community/tutorials/how-to-map-user-location-with-geoip-and-elk-elasticsearch-logstash-and-kibana.

Then, in Sense I mapped the geoip.location field to geo_point by running this:

PUT connect-log
{
    "mappings": {
      "connect-log": {
        "properties": {
          "geoip": {
            "properties" : {
              "location": {
                "type": "geo_point",
                "doc_values": true
              }  
            }
          }
        }
      }
    }
}

I think this worked because now I see this:

GET connect-log/_mapping/connect-log/field/geoip.location

{
  "connect-log": {
    "mappings": {
      "connect-log": {
        "geoip.location": {
          "full_name": "geoip.location",
          "mapping": {
            "location": {
              "type": "geo_point"
            }
          }
        }
      }
    }
  }
}

When I go to create a Tile Map visualization it detects that geoip.location is the right field to use, but it always says "No results found".

There seem to be 100's of topics with the same theme already, but it's all very convoluted especially to me!

Thanks in advance,
Ryan

2

You have geoip.location in your mappings, but your LS config has geoip.coordinates, that's why :slight_smile:

3

Thanks for the reply, I'll try changing coordinates to location. But in kibana when I look at one of the log entries I see the same value for both geoip.location and geoip.coordinates. Honestly I don't know how location gets there in the first place haha.

Here's an example of the geoip entries for one log (as seen from Kibana):

# geoip.area_code	  	###
t geoip.city_name	  	Franklin
t geoip.continent_code	  	NA
# geoip.coordinates	  	-xx.xxx, yy.yyy
t geoip.country_code2	  	US
t geoip.country_code3	  	USA
t geoip.country_name	  	United States
# geoip.dma_code	  	###
t geoip.ip	  	        xx.xx.xx.xx
# geoip.latitude	  	yy.yyy
# geoip.location	  	-xx.xxx, yy.yyy
# geoip.longitude	  	-xx.xxx
t geoip.postal_code	  	abcde
t geoip.real_region_name	Massachusetts
t geoip.region_name	  	MA
t geoip.timezone	  	America/New_York

EDIT:
I tried changing "coordinates" to "location" in my logstash config, now the geoip.location field ends up with duplicate values:

# geoip.location -xxx.xx, yyy.yy, -xxx.xx, yyy.yy

4

I got the Tile map working by reassigning the geoip.location field to one that's not nested (mylocation). I have no idea why this mattered but it seems to be working now.

    geoip {
            source => "client_ip"
            target => "geoip"
            database => "/etc/logstash/GeoLiteCity.dat"
            add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}" ]
            add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}"  ]
            add_field => [ "mylocation", "%{[geoip][location]}" ]
    }

Thanks for the replies,
Ryan

1 Like
5

Hi Ryan.

I know you've already solved this topic, but I'm experiencing the same problem. Can you clarify if following steps are correct?

1 - configure logstash with a new field during geoip filter:
(...)
add_field => [ "mylocation", "%{[geoip][location]}" ]
(...)

2 - run logstash import

3 - apply mapping for geo_point to the same index userd with logstash import

PUT connect-log
{
"mappings": {
"connect-log": {
"properties": {
"mylocation": {
"type": "geo_point",
"doc_values": true
}
}
}
}
}

4 - configure index in kibana; mylocation field must be type geo_point.

5 - create tile map visualization