Dec 21st, 2019: [EN][Kibana] Gain insights from large datasets with choropleth maps and Elastic maps

"Get It Done" is a San Diego initiative to engage citizens in reporting non-emergency problems like potholes, missed garbage collection, and street light outages. Smart phone applications make reporting easy. The "Get It Done" data set is publicly available and can be downloaded as CSV. Follow these instructions to ingest the "Get It Done" data set into Elasticsearch.

The 2019 CSV contains over 250,000 reports. Each report contains a GPS location. Lets start by viewing the reports on a map.

Viewing reports as individual markers does not provide a lot of value. Reports overlap each other, making it difficult to extract any insights from the data set.

We need a way to group reports into logical sets. We need a way to visually compare and contrast metrics across groups and detect patterns. Choropleth maps are an ideal solution. Choropleth maps shade areas in proportion to metric values and are commonly used to visualize population statistics such as race, age or political affiliation across geopolitical boundaries.

San Diego's local government is organized into 8 geographical council districts. Download the council district boundaries geojson and insert into Elasticsearch using GeoJson Upload.

Lets create a choropleth map to visualize the reports by council districts.

Each report contains a council district identifier. Add a term join to the council districts layer to join aggregated properties from the "Get It Done" data set with council district boundaries.

The council district boundaries now have two additional properties that can be used for styling.

  • Count of "Get It Done" records for the council district
  • Average "Get It Done" case age for the council district

Use data driven styling to symbolize the council district fill color by count to create a choropleth map.

Grouping and visualizing the reports by council district makes it possible to gain insights and detect patterns from the data set. It is now easy to see which council districts have the most cases. Government officials can use the insights gained by the choropleth map to more efficiently deploy scarce resources.

The "Get It Done" data set contains many dimensions. Most are not easily displayed on a map. Lets create a dashboard to gain insight into the top requested services and better understand when services are requested.

Clicking on "Dead Animal" in the tag cloud creates a filter that is applied to each panel. Now the panels only show reports requesting dead animal services. The "service request clock" panel shows that most of these service requests occur between 6 and 11 in the morning. This information can be used to ensure proper animal removal service staffing at these times.

Choropleth maps are great for visualizing large data sets. Elastic maps makes it easy to create choropleth maps that can be used along with tag clouds, bar charts, and other visualizations to gain insight from your data.