Add Geojson Files to Elastic Map Service

[m.hanna]

We have some geojson files that we created that we would like to add to our Elastic Map Service instance.
Examples -
airports.geojson that has geo points of airports worldwide
us_states_territories.geojson that includes the US territories and outlying islands (different than the US states geojson included with EMS)

I have tried to mount these geojson files to /usr/src/app/file/files and name them with the geo.json extension. Additionally, I modifed /usr/src/app/file/latest/manifest to add the new geojson files and mounted that file into the ems container.
The additional files do not show up in ems. If I mount a geojson over an existing geo.json file (ex. us_states_v1.geo.json), the shapes show up in ems and kibana, but it is less than ideal as some of the labels are off.
Is there an easy way to add our own geojson files to ems and or the ems manifest?

[jsanz (Jorge Sanz)]

Hi,

Note
Elastic does not support making changes locally to EMS File Service so take this as a technical exercise.

Why not uploading the data to Elasticsearch and provide it widely to your users in a way you can manage permissions, etc? This way of sharing data seems a bit of a stretch. You can't handle large amounts of data as you would sharing the data from your Elasticsearch database.

Anyways, let's go:

EMS file service is hosted in a public repository. If you clone and build the repo (using the NodeJS stack) you can see how the datasets from the data folder are listed in the different files in the sources folder.

To add airports to EMS you should:

• Put a valid GeoJSON file in the data folder. Ideally just with the fields you will really be using (all fields need to be described for tests to pass).
• Add a new source file that describes the dataset (see example below).
  • To get the localized labels you can run the wikidata-labels.js, in this case it would be `node scripts/wikidata-labels.js Q1248784
• Run yarn test to ensure your data and sources are valid
• Run yarn build to generate your custom EMS File Service
• Mount the dist/vector folder in your EMS Server instance at /usr/src/app/file

Source file:

{
  versions: '>= 1'
  production: true
  type: http
  note: Airports from all over the world
  wikidata: Q1248784
  data: ''
  attribution: [
    {
      label: {
        en: Made with NaturalEarth
      }
      url: {
        en: http://www.naturalearthdata.com/about/terms-of-use
      }
    }
    {
      label: {
        en: Elastic Maps Service
      }
      url: {
        en: https://www.elastic.co/elastic-maps-service
      }
    }
  ]
  fieldMapping: [
    {
      type: id
      name: gps_code
      desc: GPS code
    }
    {
      type: id
      name: iata_code
      desc: IATA code
    }
    {
      type: property
      name: name
      desc: Airport name
    }
    {
      type: property
      name: abbrev
      desc: Airport abbreviation
    }
  ]
  website: https://github.com/martynafford/natural-earth-geojson/tree/master
  license:
  {
    url: http://www.naturalearthdata.com/about/terms-of-use/
    text: Made with Natural Earth
  }
  name: airports
  legacyIds: [
    airports
  ]
  humanReadableName: {
    ar: مطار
    ca: aeroport
    de: Flughafen
    de-ch: Flughafen
    en: Airports
    en-ca: Airports
    en-gb: Airports
    es: Aeropuertos
    fr: Aéroport
  }
  emsFormats:[{
    type: geojson
    file: airports_min.geo.json
    default: true
  }]
  ttl: 108000
  weight: 2
  createdAt: "2023-12-05T15:27:27+0100"
  id: 1701786511682710992
}

The original airports file was processed using this command to remove unnecessary fields and reduce coordinates precision:

mapshaper -i airports.geo.json \
  -filter-fields iata_code,gps_code,abbrev,name \
  -o airports_min.geo.json precision=0.000001

With that I could see Airports layer both in the EMS layer list and Kibana. EMS Layers list won't render the points because it is meant to render only polygon layers, at this moment: