Geoip location not shown on Kibana map

Elastic Stack Kibana
1

I guess my problem may belong to Kibana arena.

I have been trying to figure out how to get my geo ip location show up on Kibana map. But I always get the following error:

The "metricbeat*" index pattern does not contain any of the following field types: geo_point

I have the ES 5.6 including the search, Kibana, metric beat, and file beat.

I managed to put some IP into the the metricbeat instances as env fields. The logstash was set up to accept data and put into ElasticSearch (ES).

I could see the data in geoip and I also set up the mapping for its location to be geo_point. But it kept giving me the error.

Please help!!!

Here please find the corresponding mapping (I got through GET /_template in Kibana's DevTools) and data in JSON below.

{

"metricbeat": {
"order": 0,
"template": "metricbeat-*",
"settings": {
"index": {
"mapping": {
"total_fields": {
"limit": "10000"
}
},
"refresh_interval": "5s"
}
},

......

    "properties": {
    "@timestamp": {
        "type": "date"
    },
    "geoip": {
        "type": "object",
        "dynamic": true,
        "properties": {
              "location": {
                  "type": "geo_point"
              },
             "country_name": {
                 "type": "string"
             },
             "city_name": {
                 "type": "string"
             },
             "region_name": {
                 "type": "string"
             }
        },

......

Here is the data in JSON:
{
"_index": "metricbeat-2017.10.04",
"_type": "metricsets",
"_id": "AV7pxbrMOworNTwM-ObM",
"_version": 1,
"_score": null,
"_source": {
"host_ip": "198.241.217.201",
"@timestamp": "2017-10-04T23:43:21.818Z",
"system": {
"filesystem": {
"total": 0,
"device_name": "sysfs",
"free_files": 0,
"mount_point": "/sys",
"available": 0,
"files": 0,
"used": {
"pct": 0,
"bytes": 0
},
"free": 0
}
},
"geoip": {
"country_name": "United States",
"city_name": "San Francisco",
"region_name": "California",
"location": {
"lon": -122.3802,
"lat": 37.6198
}
},
"beat": {
"name": "wzhang_EPSim_21",
"hostname": "wzhang_EPSim_21",
"version": "5.6.2"
},

2

Modifying the vendor supplied metricbeat template looks fragile to me, but when I did it it worked once I refreshed the fields under Management / Index Patterns.

3

Make sure that you've refreshed your Index Pattern in Kibana after modifying the mappings to include those fields. Using the following two queries via the DevTools, I've been able to create data that can be used on the Tilemap in 5.3.0:

PUT geopoints
{
  "mappings": {
    "default": {
      "properties": {
        "@timestamp": {
          "type": "date"
        },
        "geoip": {
          "type": "object",
          "dynamic": true,
          "properties": {
            "location": {
              "type": "geo_point"
            },
            "country_name": {
              "type": "string"
            },
            "city_name": {
              "type": "string"
            },
            "region_name": {
              "type": "string"
            }
          }
        }
      }
    }
  }
}

POST geopoints/default
{
  "@timestamp": "2017-10-05T10:00:00Z",
  "geoip": {
    "country_name": "United States",
    "city_name": "San Francisco",
    "region_name": "California",
    "location": {
      "lon": -122.3802,
      "lat": 37.6198
    }
  }
}
4

I did DELETE /metricbeat-YYYY.MM.DD in DevTools to delete data from ES and restarted LogStash, ES, MetricBeat earlier. But none of them worked!

One thing I am not sure if I had done earlier was to delete or refresh the metricbeat* index pattern in Kibana.

Now, I just trashed the index pattern and created the index pattern. Wu La La, it showed up. So, for those who have the same kind of issue, remember to recreate the index pattern in Kibana, not just recreate the index in ES. I thought I needed to recreate the index in the data not the pattern in Kibana.

But the problem now is that my dashboard can not be loaded any more. They used to show numbers and charts before I recreated the index pattern.

Here is the msg:

Could not locate that index-pattern (id: metricbeat-*), click here to re-create it

5

OK. Solved my own problem. Learned it in a hard way.

I recreated the index pattern in Kibana to get the geo location worked. But the dashboard charts used to work fine did not work any more and had the error in the previous message.

I even followed its suggestion to recreate the index pattern again a few times. It did not help.

Eventually, I deleted those dashboard charts and imported all charts, which I downloaded from MetricBeat site earlier, back in there again. Whooosh, it worked!

I guess when the charts were loaded they were somehow bound to those index pattern found in ES for Kibana. When I recreated the index pattern and deleted the old ones, that bound was broken. It could not automatically transferred to the new index pattern even though they are pretty much the same except the geo_point. So, every time when you recreate your index pattern, you'd better delete all your charts and import them again.

A feedback to development team: would it be a good idea to somehow allow users change charts underline index pattern to a new one?

Thanks,

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.