This post is also available in Spanish
Introduction
One of the most well-known security practices is to never expose your Elasticsearch cluster to the Internet. But we are in the Christmas season, and we like to be festive and fun, so I suggest this technical exercise of accessing your favorite database and search engine directly from your browser for nothing less than rendering some fancy maps.
As you may know, Elasticsearch can render large amounts of geospatial data leveraging vector tiles. If you don't know about this, please read this excellent introduction by Ignacio and Thomas.
Like many other Elasticsearch endpoints, the Vector Tiles API expects a JSON payload to run arbitrarily complex search and aggregations, but that's not how map browser libraries work. They expect a simple URL with the zoom, X, and Y coordinates as in https://my-server/tiles/${z}/${x}/${y}. The classic approach (for this and many other reasons) is to build a middleware software that crafts the correct queries for Elasticsearch and serves as a facade. Hence, client applications send regular GET requests with parameters without knowing anything about the backend.
Sequence diagram with the classic approach to rendering vector tiles.
If you are learning about Elasticsearch, building something straightforward, or having fun, we can remove that middleware and let our browser talk directly with the database. But how?
MapLibre is a popular webmapping library that leverages the latest browser technologies to render vector tiles with beautiful basemaps. That is what we use at Kibana Maps, and we love the project and the hard work behind it. The library has a fancy feature that allows customizing the request to your backend to your needs, even changing the type from a GET to a POST, and setting your headers and payload, so let's use it!
Sequence diagram of the Christmas-YOLO approach.
Data
The Overture Maps Foundation is a joint effort between Microsoft, Meta, Esri, and others to collect and publish large open datasets. For this exercise, I uploaded the Overture 2023-07-26-alpha.0 places theme, with more than 59 million worldwide locations by Meta and Microsoft.
You can explore the dataset on this Kibana dashboard
Overture Maps Foundation Places theme dashboard.
Setting up Elasticsearch
To access this data from the browser, we need to first allow some CORS headers in our database as detailed in the workshop data preparation section. Then, as described in the following section in the workshop, we need to create a read-only API key to access this index to make queries to the /overture-places search and vector tiles endpoint.
Tip
If you follow along with the workshop, please don't close your Kibana window because it will be handy to explore the data, see the fields available, their domains, and so on.
Viewing Christmas locations
I left the sample code for this post on the GitHub repository jsanz/advent-2023 and the running demo at https://jsanz.github.io/advent-2023. The code is as simple as it can be: just an HTML document with a JavaScript module. The basemap on this example is served by MapTiler and their winter style.
Default map showing locations with "Santa Claus" names.
The exciting bits are at the very beginning of app.js script, specifying the Elasticsearch endpoint, the data API key, and finally the getDynamicTransformRequest function. This function is passed when the map is instantiated for the transformRequest creation option. This function will check for every data request, and for those that point to our Elasticsearch host, it will attach a payload with our search configuration and API key in a header.
const getDynamicTransFormRequest = function (url, resourceType) {
/* This function enriches the HTTP request to include
the ES search body, change to a POST request, and include
the Content-Type header */
if (resourceType == "Tile" && url.startsWith(ES_HOST)) {
const query = document.getElementById("search").value || DEFAULT_QUERY;
/* Build a VT query payload */
const body = {
grid_precision: 0,
exact_bounds: true,
extent: 4096,
fields: ["name", "category_main", "category_alt", "source", "social"],
query: {
bool: {
filter: [
{
multi_match: {
lenient: true,
query: `*${query}*`,
type: "phrase",
},
},
],
},
},
}
return {
url: url,
method: "POST",
headers: {
"Content-Type": "application/json",
Authorization: `ApiKey ${ES_APIKEY}`,
},
body: JSON.stringify(body),
};
}
};
[...]
const map = new maplibregl.Map({
container: "map",
style,
center: [0, 15],
zoom: 1,
hash: true,
transformRequest: getDynamicTransFormRequest,
});
The rest of the script is regular MapLibre code to render the locations based on a property and attaching an event to render a popup when the user clicks on a location, allowing you to learn more about that point of interest.
Christmas Tree Farms are super popular on the East Coast of the US.
Tip
Feel free to open the browser console to explore the requests to Elasticsearch.
The first time the page loads, it places a default phrase in the query to search for "Santa Claus" locations. I left a few suggestions for that input, but feel free to get creative and find other fancy Christmas spots.
Hari Natal everyone!
Getting Complicated
This exercise was the first step covered in the workshop. I invite you to follow the rest of the lessons to learn how to search and render individual documents and, more interestingly, aggregate your data to summarize and get insights for millions of records with our geohex aggregation.
Geohex aggregation for NYC 311 data.
Tip
If you don't want to do the full workshop and want to play around with the maps, visit https://webmapping-elasticsearch-2023.glitch.me for a deployed instance of the workshop code.
A final suggestion: the Kibana Inspect tool is invaluable when developing Elasticsearch applications and is also available in the Maps application. Use it to learn how the app mixes all the data from the different UI elements, like the Kibana search bar and filters, to generate the final requests sent to the backend.
Kibana Maps inspect tool.
Conclusion
With the little MapLibre trick detailed above, we can explore and learn the Elasticsearch DSL nicely, with a fast, interactive loop and improved developer experience: update your query or rendering code and reload your browser 
. It also gives us a glimpse of the fantastic Elasticsearch capabilities combined with geospatial features for high-performance big data applications.
Happy Holidays!!