What does Coordinate Map Visualization read?

Elastic Stack Kibana
1

Hi,

So I have mapped some ID to [geoip][location][lon] and [geoip][location][lat] and the data is showing up well, but the Coordinate Map Visualization is still not showing any data.
Is the visualization reading more data than long and lat? If then what would they be? Please advise...

filter:

    translate {
      field => "DomicileTransit"
      dictionary_path => '/etc/logstash/config/CustomerConnect_mapping.yaml'
    }
    mutate {
      gsub => [ "translation", " ", "" ]
      split => {"translation" => ","}
      add_field => ["[geoip][location][lat]","%{[translation][0]}"]
      add_field => ["[geoip][location][lon]","%{[translation][1]}"]
      convert => { "[geoip][location][lat]" => "float" }
      convert => { "[geoip][location][lon]" => "float" }
   }
2

Could you share your mapping? The only other thing that would possibly filter the results would be the time range. Could you also share a screenshot of the map? Thanks!

3

Hi,

Below is the mapping dict:

---
"0001": "45.505083,-73.557775"
"0002": "43.648799,-79.380626"
"0003": "49.895875,-97.137989"
"0004": "49.285904,-123.119465"
"0005": "45.419168,-75.698501"
"0006": "43.256429,-79.874934"
"0007": "46.804237,-71.22071"
"0008": "42.98502,-81.25042"
"0009": "44.648767,-63.572943"
"0010": "51.046914,-114.069837"
"0011": "45.272674,-66.062056"
"0012": "48.42605,-123.364676"
"0013": "50.450228,-104.611147"
"0014": "53.542539,-113.493337"
"0015": "43.14015,-80.263316"
"0016": "44.229732,-76.48155"
"0017": "44.3049,-78.319081"
"0018": "42.318138,-83.039155"
"0019": "46.139311,-60.18438"
"0020": "45.425794,-75.716641"
"0021": "46.196147,-59.957574"
"0022": "48.406076,-89.24043" --List goes on

I also tried trimming to 2 significant digit but still not working:

---
"0001": "45.50,-73.55"
"0002": "43.64,-79.38"
"0003": "49.89,-97.13"
"0004": "49.28,-123.11"
"0005": "45.41,-75.69"
"0006": "43.25,-79.87"
"0007": "46.80,-71.22"
"0008": "42.98,-81.25"
"0009": "44.64,-63.57"
"0010": "51.04,-114.06"
"0011": "45.27,-66.06"
"0012": "48.42,-123.36"
"0013": "50.45,-104.61"
"0014": "53.54,-113.49" --List goes on

Please see screenshots FYI. Note that index and timeframe are the same. I didn't touch anything, so there must be data there. Just somehow geoip.location not read properly...

discover
discover.jpg1920×1080 124 KB
GeoMap
GeoMap.jpg1366×768 158 KB

4

Thanks for sharing... Looks like things are mapped correctly.

At the bottom left corner of the map is a little button that will show some data about the request including a table of the results. Could you expand that and take screenshots of the results table there?

5

The response seems not having [lon] and [lat] data in them... In fats, [hit] is empty. Are they supposed to be empty?

Request:

{
  "size": 0,
  "query": {
    "bool": {
      "must": [
        {
          "match_all": {}
        },
        {
          "range": {
            "@timestamp": {
              "gte": 1537422208609,
              "lte": 1537465408609,
              "format": "epoch_millis"
            }
          }
        }
      ],
      "must_not": []
    }
  },
  "_source": {
    "excludes": []
  },
  "aggs": {
    "2": {
      "geohash_grid": {
        "field": "geoip.location",
        "precision": 2
      },
      "aggs": {
        "3": {
          "geo_centroid": {
            "field": "geoip.location"
          }
        }
      }
    }
  }
}

Response:

{
  "took": 4,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 3,
    "max_score": 0,
    "hits": []
  },
  "aggregations": {
    "2": {
      "buckets": []
    }
  },
  "status": 200
}
6

Hi @lukas

Any updates?
Can I know what's wrong with the visualization?

Regards,
Perry

7

Sorry for the delay, I'm still not sure what's happening here. Could you go to dev tools and do

GET */_mapping

and paste the response here?

8

Hi Lukas,

I took a look at the mappings, seems like the lon and lat fields are text i/o floats... Is that normal?
If not, how are we fixing this?

{
  "dts-2018.09.20": {
    "mappings": {
      "log": {
        "properties": {
          "@timestamp": {
            "type": "date"
          },
          "@version": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          ...
          "geoip": {
            "properties": {
              "location": {
                "properties": {
                  "lat": {
                    "type": "text",
                    "fields": {
                      "keyword": {
                        "type": "keyword",
                        "ignore_above": 256
                      }
                    }
                  },
                  "lon": {
                    "type": "text",
                    "fields": {
                      "keyword": {
                        "type": "keyword",
                        "ignore_above": 256
                      }
                    }
                  }
                }
              }
            }
          },
          ...
        }
      }
    }
  }
}
9

Seems like you'll have to update your elasticsearch mapping to use the geo_point datatype. Are you familiar with the PUT _mapping command?

10

Hi Lukas,

Not really...
I looked that up and I'm assuming it's following:

PUT dts-2018.09.20
{
"mappings": {
"_doc": {
"geoip": {
"properties": {
"location": {
"type": "geo_point"
}
}
}
}
}
}

Can you correct me if I'm wrong? Also, am I able to put mapping to index patterns instead of indices? Or do I need to create a template?

Thanks,
Perry

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.