Routing documents dynamically from spark connector

Elastic Stack Elasticsearch
1

I want to be able to do some thing like.

PUT my_index/_doc/1?routing=user1&refresh=true
{
"title": "This is a document"
}

As described in https://www.elastic.co/guide/en/elasticsearch/reference/6.2/mapping-routing-field.html#mapping-routing-field.

Does the connector support it. So I may have a dataframe with 10000 documents and they might belong to 4 distinct users and I want to route the related docs to a shard/shards.

2

You can specify the routing on a per document basis by referencing the field to extract the routing information from via the es.mapping.routing setting: https://www.elastic.co/guide/en/elasticsearch/hadoop/current/configuration.html#cfg-mapping