Trasnform - number of shards defined in the template is not applied

Elastic Stack Elasticsearch
1

Hi,

I have defined an index template with 4 shards, ILM policy and fields mapping. When I execute my Transform, every thing defined in the template is applied except the number of shards. My ElasticSearch version is 7.6.2. I know that Transform is in beta version on 7.6.2, is this a bug or an expected thing ?

By the way, my actual workaround is to create manually my index with the wanted number of shards before executing the Transform.

Regards,
Hamza

2

Hi @Hamza_BENNANI

This is expected behaviour and your workaround is recommended if you need to apply alternative mappings. We updated recent documentation to reflects this.
https://www.elastic.co/guide/en/elasticsearch/reference/current/put-transform.html (although this is not in the 7.6 docs).

Best wishes
Sophie

3

Hi @sophie_chang,

Thank you for your quick answer. Could you please explain why it's expected to have only 1 Primary shard?

By the way, for performances reasons, I need to set more than 1 Primary shard.

Regards,
Hamza

4

Hi @Hamza_BENNANI

There is a lot of discussion regarding default of 1 shard here https://github.com/elastic/elasticsearch/pull/30539

Please use the workaround to create the destination index with a higher shard count.

If you upgrade to 7.7+ the transforms preview endpoint will also return the generated mappings, making it easier to see the deduced mappings and index settings. You then have the option to override by manually creating the index instead.

Hope this helps.
Sophie