(Elasticsearch NEST C#) How to apply CopyTo to any field in an index

[WestCoastWondering (Chris)]

Hi,

My team is using Elasticsearch NEST to implement search within a .Net application.

The documents in this index include static text fields, such as title and description as well as dynamic fields such as field1234.

I want to use the CopyTo functionality to copy all of the field contents to a particular field (let's call it allContent) during indexing, so that I can search efficiently later.

I read this relevant discussion:

Copy_to in Nest Attributes

You can't specify copy_to fields with attribute mapping because multiple fields can be specified, which are represented with the Fields type, and attribute parameters are restricted to constant values, so it's not possible to assign an instance of Fields within an [Attribute] declaration. You can define copy_to fields with fluent mapping though. Since the original _source field is not modified, you don't need to add another property to your C# POCO for a copy_to field. Here's an example client…

In that discussion's example by @forloop , the field to copy is explicitly named in the .properties .text portion. Is it possible to broaden this to all fields?

[WestCoastWondering (Chris)]

This was first (feeble) attempt at using dynamic templates for this purpose:

EsClient.CreateIndex(name, c => c
                    .InitializeUsing(indexConfig)
                    .Mappings(m => m.Map<AppBuilder>(mp => mp.AutoMap()
                        .DynamicTemplates(dts => dts
                            .DynamicTemplate("all_strings", dt => dt
                                .MatchMappingType("string")
                                .Mapping(dtm => dtm
                                    .Text(dtmt => dtmt
                                        .Index(true)
                                        .CopyTo(ct => ct
                                            .Field("_ternarySearch"))
                                    )
                                )
                            )
                         )
                        .Properties(ps => ps
                            .Completion(cn => cn
                                .Name(p => p.Suggest)))))

It did not seem to work (generating content into a field named _ternarySearch for the documents)

[forloop (Russ Cam)]

Dynamic templates is the approach to take to be able to configure copy_to to another field or fields for dynamic fields i.e. fields whose names you don't know ahead of time and can provide explicit mapping for.

Your example looks good and produces the following request (I'm assuming you're using NEST 6.x based on the CreateIndex() method call)

{
  "mappings": {
    "appbuilder": {
      "dynamic_templates": [
        {
          "all_strings": {
            "mapping": {
              "type": "text",
              "copy_to": [
                "_ternarySearch"
              ],
              "index": true
            },
            "match_mapping_type": "string"
          }
        }
      ],
      "properties": {
        "suggest": {
          "type": "completion"
        }
      }
    }
  }
}

This will map new string fields within the _source document as text field mapping, index the results of running the field value through analysis against the field name in the inverted index ("index": true, which is the default), and also copy the field value to "_ternarySearch" field, index the results of running the field value through analysis and index them in the inverted index against the "_ternarySearch" field.

Note that the original _source document sent to Elasticsearch is not changed by this process, and is persisted verbatim as it was sent in, meaning that the _source returned in the hits for documents matching a query will not contain a "_ternarySearch" field, because it didn't exist on _source.

[WestCoastWondering (Chris)]

Thank you for the detailed description. It did throw me for a loop when I didn't see the results in _source, but after I re-read your explanation, it makes sense.

I'm getting the results I expect now for dynamic fields.

Much appreciated!