Geopoint Coordinate map question

[alexwbai (Alex)]

Hi,

Rather than have a bazillion messages go to Elasticsearch, I'd like to summarize the number of people in a subway station in a json record at a given time like this. A JSON event sent to Elasticsearch would look something like this (Excuse the JSON syntax; I'm freestyling here):

{
stationName: "Wellesley St.",
lat: 78.9348,
long: -98.8764,
peoplecount: 50,
@timestamp: 15098796,
}

I would send something like this for each station every couple of seconds and use a coordinate map vizualization to tell me show me many people are in the subway station based on a SUM of the "peoplecount" field. (eg. The higher the count of people, the bigger the dot on the map)

Is this possible with Kibana? Or do I need to send every separate event to Elasticsearch and then do a count.

Thanks,
AlexW

[warkolm (Mark Walkom)]

Either will work.
Individual events also means you could do things like track journeys. But the flipside is it's more data, you could do _rollups on the data after a period of time (eg a month) to reduce long term storage though.

Just make sure you map the event in a format that Elasticsearch understands - https://www.elastic.co/guide/en/elasticsearch/reference/6.5/geo-point.html

[alexwbai (Alex)]

Hey Mark,

Thanks so much for taking the time to reply! I was hoping you could guide me a little further if possible (with Kibana specifically).

So I have the data going to Elasticsearch and the geocoordinates seem to be working fine. Here's what the sent data looks like (I'm using the bulk api):

{"id": "672d4f9712db446888c13bbaccfbfb7c", "ts": 1547821765000, "station": "spadina", "devicecount": 101, "location": {"lat": 43.6694, "lon": -79.40440000000001}}
{"create": {"_index": "test", "_type": "_doc", "_id": "ed6e18ef0c66423e8778da15e263f050"}}

{"id": "ed6e18ef0c66423e8778da15e263f050", "ts": 1547821765000, "station": "spadina", "devicecount": 101, "location": {"lat": 43.6678, "lon": -79.4042}}
{"create": {"_index": "test", "_type": "_doc", "_id": "403bbe951d524127876d61cfb79328c4"}}

And this is being done for every station (about 80 stations). I'm streaming this into ES successfully.

My goal is to plot these locations on a geocoordinate map and then do a SUM of the "devicecount" grouped by station. The end result will be a map of Toronto which is refreshing periodically with how many devices are seen in each station.

I'm having difficulties in how I can group the stations in the geocoordinate map. I only seem to get one coordinate with a giant sum of everything. Any thoughts?

AlexW

[warkolm (Mark Walkom)]

What is the current mapping for the index?