I'm trying to get my head around the new concept of component templates in ES7.9 before I start upgrading from ES6.8.
My problem is that I've got a dozen index templates where I use the "order" field to build up the mapping I want for each index type, by applying the general settings and mappings in the lowest order (0) template and the more specialized settings and mappings in the higher order. Now, in ES7 the "order" field is gone, replaced by "priority" and "composed_of". This turns my world view on its head, instead of adding more and more specialized templates to generate the precise index mapping I can now only use one template per index type (since only the one of highest priority is used). Thus I'm trying to figure out a way to use component templates.
As there isn't any official documentation on their usage and limitations (as far as I can tell after searching on elastic.co) I've had to build some test cases in a ES7.9 test cluster to see how I can build up the existing index templates using component templates and composed_of.
My plan was to create dedicated components with filters and analyzers for my various use cases and then other components with the actual document field mapping, as I need to maintain the strict mapping I have in the ES 6.8 clusters. And because our documents can have up to a hundred fields I prefer to create a general document mapping component plus a few smaller and more specialized component templates that I can include in the relevant index templates using the composed_of field.
However, it seems I can't define my analyzers and filters in a separate component template. I managed to install those components in a test cluster but when I try to install the component with the document mappings it fails because it can't find the filters and analyzers in the other component.
Here's the test I did. An analyzer is defined in one component template that I successfully installed in the test cluster and can get like this:
component-get.sh test_byline
{
"component_templates" : [
{
"name" : "test_byline",
"component_template" : {
"template" : {
"settings" : {
"index" : {
"analysis" : {
"analyzer" : {
"byline_field" : {
"filter" : [
"lowercase"
],
"type" : "custom",
"tokenizer" : "icu_tokenizer"
}
}
}
}
}
}
}
}
]
}
Then I tried to use this "byline_field" analyzer in a test document mapping (this is just a test, the final mapping will have a hundred fields):
{
"template" : {
"mappings" : {
"dynamic" : "strict",
"properties" : {
"byline" : {
"analyzer" : "byline_field",
"fields" : {
"raw" : {
"index" : true,
"type" : "keyword"
}
},
"index" : true,
"type" : "text"
}
}
}
}
}
But when I tried installing this component template I got:
Failed to parse mapping [_doc]: analyzer [byline_field] not found for field [byline]
So it seems clear that the component_template API parses the new template and fails to locate the analyzer "byline_field" which was previously installed with the other component template.
My question then: Is it possible in one component template to refer to filters or analyzers in another, or must the document mapping, filters and analyzers be defined within the same component template?
If the latter is the case I will have to rethink how I build up the index templates.
I could of course add the analyzers to the document mapping template, but then I'll run into problems if I use composed_of with two or more document mapping templates that both define the same filters and analyzers. And that would be a show stopper. Plus I don't like the extra maintenance of defining the same set of analyzers and filters (some are very large) in multiple component templates.
One solution would be to write my own index-template maker which would take a set of source files and a rule / meta template to build each index template on the fly. Then I can skip the component templates altogether and just build new index templates whenever changes are needed.
Hopefully someone out there has tackled this problem by now 
Cheers,
bernt