# How to specify routing for index transform job

**URL:** <https://discuss.elastic.co/t/how-to-specify-routing-for-index-transform-job/292393>\
**Category:** Elasticsearch\
**Created:** [December 19, 2021, 9:07am UTC](https://discuss.elastic.co/t/how-to-specify-routing-for-index-transform-job/292393 "2021-12-19T09:07:51Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Assaf\_Cohen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/assaf_cohen/32/46023_2.png) [@Assaf\_Cohen](https://discuss.elastic.co/u/Assaf_Cohen)\
**Post date:** [December 19, 2021, 9:07am UTC](https://discuss.elastic.co/t/how-to-specify-routing-for-index-transform-job/292393/1 "2021-12-19T09:07:51Z")

</div>

Hello,

1. we have an index transform job, created as below (shortened for the sake of brevity)

```auto
PUT /_plugins/_transform/my_test_transform
{
  "transform": {
    "enabled": true,
    "schedule": {
      "interval": {
        "period": 1,
        "unit": "Minutes",
        "start_time": 1602100553
      }
    },
    "description": "create index my_test_transform",
    "source_index": "my_test_src_index",
    "target_index": "my_test_target_index",
    "page_size": 1,
    "groups": [
      {
        "terms": {
          "source_field": "message_header.global_tenant_id",
          "target_field": "message_header.global_tenant_id"
        }
      }
    ],
    "aggregations": {
      "usage_data.data_read_kb": {
        "sum": {
          "field": "usage_data.data_read_kb"
        }
      },
      "usage_data": {
        "scripted_metric": {
          "init_script": "state.docs = [];",
          "map_script": """ 
          
            Map span = [
              'occurred_message_timestamp_utc_ms':doc['occurred_message_timestamp_utc_ms'].value
            ];
            state.docs.add(span);
          """,
          "combine_script": "return state.docs;",
          "reduce_script": """ 
            def ret = new HashMap();
            ret['device_friendly_name'] = 'test123';
            return ret;
          """
        }
      }
    }
  }
}

```

1. Both source and target indexes contain multi tenant documents, so we use document field "message\_header.global\_tenant\_id" as routing value.  
The source and target indexes both have an index template with mapping that specifies:

```auto
    "_routing": {
		"required": true
	}

```

1. when running the transform job it fails with exception "RoutingMissingException"
2. to verify this is the root cause, we removed the "routing = true" from target index template, and now the transform job completes successfully!!
3. However we would like to keep "routing = true" in target index, so that tenant documents in target index are located in same shard (better for search performance)

How do we accomplish this?

Thanks,  
Assaf

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [December 20, 2021, 7:46am UTC](https://discuss.elastic.co/t/how-to-specify-routing-for-index-transform-job/292393/2 "2021-12-20T07:46:39Z")

</div>

Welcome to our community! 😃

This isn't Elasticsearch, as the transforms API is actually `_transform/`. What are you using here that is a plugin?

---

<div class="post-metadata">

**Author:** ![Assaf\_Cohen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/assaf_cohen/32/46023_2.png) [@Assaf\_Cohen](https://discuss.elastic.co/u/Assaf_Cohen)\
**Post date:** [December 20, 2021, 9:56am UTC](https://discuss.elastic.co/t/how-to-specify-routing-for-index-transform-job/292393/3 "2021-12-20T09:56:15Z")

</div>

thank you!  
you are right, we are currently running a few POCs most of them on elastic and others on opensearch. I accidently pasted the opensearch example.

How would this be accomplished this on Elasticsearch?  
I assume we need to somehow update the painless script above to use field "message\_header.global\_tenant\_id" as routing value?  
or maybe update the target index template, which we create in advance, to use "message\_header.global\_tenant\_id" as the default routing field?

thanks,  
Assaf

---

<div class="post-metadata">

**Author:** ![Hendrik\_Muhs](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hendrik_muhs/32/25802_2.png) [@Hendrik\_Muhs](https://discuss.elastic.co/u/Hendrik_Muhs)\
**Post date:** [December 20, 2021, 10:57am UTC](https://discuss.elastic.co/t/how-to-specify-routing-for-index-transform-job/292393/4 "2021-12-20T10:57:09Z")

</div>

Elasticsearch transform has a different implementation to the opensearch one or rather vice versa. Anyway, you find the docs for Elasticsearch transforms [here](https://www.elastic.co/guide/en/elasticsearch/reference/current/transforms.html). Not knowing your use case, I assume the continuous mode might be of interest for your application.

ES Transform itself does not provide a configuration for `_routing`, however this is not a limitation, because you can use an ingest pipeline as output of an Elasticsearch transform (look for the `pipeline` option under `dest`). Ingest pipelines provide great flexibility and you can e.g. change `_index` or `_routing` using a set processor.

---

<div class="post-metadata">

**Author:** ![Assaf\_Cohen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/assaf_cohen/32/46023_2.png) [@Assaf\_Cohen](https://discuss.elastic.co/u/Assaf_Cohen)\
**Post date:** [January 2, 2022, 2:45pm UTC](https://discuss.elastic.co/t/how-to-specify-routing-for-index-transform-job/292393/5 "2022-01-02T14:45:22Z")

</div>

thank you!  
I will try using ingest pipeline as output of an Elasticsearch transform

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 30, 2022, 2:45pm UTC](https://discuss.elastic.co/t/how-to-specify-routing-for-index-transform-job/292393/6 "2022-01-30T14:45:27Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.