Having written meticulous JSON Schema files, using the Elastic Common Schema (ECS) guidelines, describing our events I am surprised to find no obvious way to import and use those schemas in Elastic Stack. Am I missing something?
Hello and welcome,
It is not clear what you mean with import JSON Schema for ECS Events, can you provide more context?
ECS is a common schema for naming and mapping fields to help you normalize all your data.
With the schema you can create indices templates that will be applied to your indices while ingesting data.
Schemas turn data into information. ECS helps with that, I can promote embedded event information attributing a User to ECS, for example. What in the Elastic Stack, or ecosystem, would use our event specific schemas?
The indices where your data will be stored.
In elasticsearch every document will be part of an index and every index needs to have a mapping (schema), in this mapping you will define which data type each field has.
If you do not create a mapping for your index defining the data type of your fields, then Elasticsearch per default will try to infer the data type on the first time it receives a document for that specific field.
You can create a mapping manually using an API or automatically using a template.
One common issue is that different sources of data could have different field names for the same entity, like userName or user_name or usr, one of the goals of the Elastic Common Schema is to serve as a guide on how a field for the user name entity should be named and mapped.
In this case it defines that it should be named user.name and mapped as a keyword as described here in the documentation.
But for example if you have an event where you have a field named userName, you will need to normalize it before indexing the data in elasticsearch.
Not sure if I got exactly what is your question/issue you didn't provide any example of the json schema you created nor what you are trying to do.
When the data is already structured with an available data model schema, as ours is, you want to use that machine readable file to guide that mapping.
We designed our schema on the ECS base, so we've explicitly broken out user.name for example. The available JSON Schema contains description, type, format, enum, etc. which are rich definitions of all the contained data.
I'm not hearing that we can use this in any way other than someone manually reading it and using that to inform the index mapping. Ideally it would be used during search and analysis where the user could be guided by the explanatory text descriptions and types (e.g. enumerations, ranges).
Yeah, I'm sorry, but it is not clear what exactly you have, what is your json schema as you didn't share anything about it, and what you want to do.
If you provide some example of what you have and what you want to do, it would be easier to know if this is possible with elasticsearch or not.
Yeah, there is nothing like this in Elasticsearch.