# Custom ECS to Index Template **URL:** \ **Category:** Elasticsearch\ **Tags:** ecs-elastic-common-schema, ilm-index-lifecycle-management\ **Created:** [August 19, 2020, 3:31pm UTC](https://discuss.elastic.co/t/custom-ecs-to-index-template/245638 "2020-08-19T15:31:05Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![jakobdoering](https://avatars.discourse-cdn.com/v4/letter/j/ac8455/32.png) [@jakobdoering](https://discuss.elastic.co/u/jakobdoering)\ **Post date:** [August 19, 2020, 3:31pm UTC](https://discuss.elastic.co/t/custom-ecs-to-index-template/245638/1 "2020-08-19T15:31:05Z")
Hi, we extended ECS for a specific use case where logstash is used to read some CSV, filter the data to match the custom ECS Extension and then send the data to elasticsearch. I am now wondering how to transfer my ECS extensions into an Index Template. Creating it manually in Kibana works for this single case but i plan on using the extension on more occasions. Is there some preexisting tool to create an Index Template from a custom ECS version or do i have to build my own tooling for this use case (leveraging the api to create the index template from the generated custom ecs mapping)? Kind Regards Jakob ---
**Author:** ![ebeahan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ebeahan/32/78989_2.png) [@ebeahan](https://discuss.elastic.co/u/ebeahan)\ **Post date:** [August 19, 2020, 4:14pm UTC](https://discuss.elastic.co/t/custom-ecs-to-index-template/245638/2 "2020-08-19T16:14:43Z")
Welcome to the community @jakobdoering! In addition to the published ECS specification, the [ECS GitHub repo](https://github.com/elastic/ecs) also provides tooling for this type of use case. I'd suggest getting started [here](https://github.com/elastic/ecs/blob/master/USAGE.md) by looking through the ECS tooling usage guide. The guide provides instructions on downloading, installing dependencies, and getting started. The [`--include` option](https://github.com/elastic/ecs/blob/master/USAGE.md#include) allows users to include their own custom schema definitions which are then merged with the official ECS fields. Once you run the generator with `--include`, ES index templates will be included in the generated artifacts: ```auto $ python scripts/generator.py --include ../myproject/custom-fields/ --out ../myproject/out/ ``` The generated ES templates would be found in your `../myproject/out/generated/elasticsearch` directory. These templates will include both the ECS fields and your custom fields definitions. ---
**Author:** ![jakobdoering](https://avatars.discourse-cdn.com/v4/letter/j/ac8455/32.png) [@jakobdoering](https://discuss.elastic.co/u/jakobdoering)\ **Post date:** [August 20, 2020, 6:27am UTC](https://discuss.elastic.co/t/custom-ecs-to-index-template/245638/3 "2020-08-20T06:27:21Z")
I already did the generation process but i am not quite sure if the only possibility to add the generated mapping to an index template/ index is doing it manually. The Beat modules create their own index templates and add the mappings themselves. Is the module which is doing that task available outside of beats? ---
**Author:** ![ebeahan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ebeahan/32/78989_2.png) [@ebeahan](https://discuss.elastic.co/u/ebeahan)\ **Post date:** [August 20, 2020, 2:30pm UTC](https://discuss.elastic.co/t/custom-ecs-to-index-template/245638/4 "2020-08-20T14:30:45Z")
Similar to Beats, the Elasticsearch output plugin in Logstash has an option available to [specify your custom index template](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-template) in the config. The ECS tooling which generates custom mappings doesn't include built-in support for uploading them today. The generated mapping can be manually uploaded using the index template API ([example](https://github.com/elastic/ecs/tree/master/generated/elasticsearch#instructions)) or through [Index Management](https://www.elastic.co/guide/en/kibana/current/managing-indices.html) in Kibana. ---
**Author:** ![webmat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/webmat/32/46191_2.png) [@webmat](https://discuss.elastic.co/u/webmat)\ **Post date:** [August 20, 2020, 3:46pm UTC](https://discuss.elastic.co/t/custom-ecs-to-index-template/245638/5 "2020-08-20T15:46:50Z")
The process linked to, based on the ECS scripts is ideal for fully custom indices. When adding fields to Beats indices it's more tricky. The Beats do follow ECS, but they also define hundreds of additional custom fields that aren't in ECS. So if you're adding fields on top of Beats modules, you'd have to look for guidance on this in the Beats docs. Note that I'm not sure there _is_ documentation on how to do that for Beats. Every time I look, I come up empty. Perhaps you can ask about that in a Beats discuss post. ---
**Author:** ![jakobdoering](https://avatars.discourse-cdn.com/v4/letter/j/ac8455/32.png) [@jakobdoering](https://discuss.elastic.co/u/jakobdoering)\ **Post date:** [August 21, 2020, 7:57am UTC](https://discuss.elastic.co/t/custom-ecs-to-index-template/245638/6 "2020-08-21T07:57:22Z")
I was also thinking about using component templates which were just introduced with the 7.9 release. Any thoughts on that? ---
**Author:** ![ebeahan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ebeahan/32/78989_2.png) [@ebeahan](https://discuss.elastic.co/u/ebeahan)\ **Post date:** [August 21, 2020, 3:31pm UTC](https://discuss.elastic.co/t/custom-ecs-to-index-template/245638/7 "2020-08-21T15:31:29Z")
Component templates will certainly help solve for reusing settings, mappings, and aliases across multiple index templates if that's a need you require. The experimental Elasticsearch templates generated by the ECS tools today are still based on the legacy index template. ---
**Author:** ![webmat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/webmat/32/46191_2.png) [@webmat](https://discuss.elastic.co/u/webmat)\ **Post date:** [August 31, 2020, 4:05pm UTC](https://discuss.elastic.co/t/custom-ecs-to-index-template/245638/8 "2020-08-31T16:05:54Z")
@jakobdoering for current Beats and modules the component templates won't help you: I think only the data sources in Elastic Agent are using component templates. But I did find the documentation for adding fields to Beats templates, which you can use right now. The config option is `setup.template.append_fields ` and is documented at [https://www.elastic.co/guide/en/beats/filebeat/current/configuration-template.html](https://www.elastic.co/guide/en/beats/filebeat/current/configuration-template.html) Note that when adding fields this way, you'll have to run your Beats setup again, so the index template is updated. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 28, 2020, 4:06pm UTC](https://discuss.elastic.co/t/custom-ecs-to-index-template/245638/9 "2020-09-28T16:06:10Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.