Adding copy_to support for copying data from object to nested type


(kailoofi) #1

tl;dr Looking at the ES code 5.x, it seems possible to support copy_to from object to nested type. My team is interested in contributing this feature back into ES but before that we want to understand the reason for not having this support from the start? Was it because of the complex implementation? Or any other technical limitations that we might not have accounted for?

Long version
My team uses Elasticsearch copy_to functionality to copy and aggregate data between fields in our mappings. Recently, we have been exploring a mapping design where we needed the same json data in both the object type and nested type fields to support co-related as well as uncorrelated search scenarios.

We started exploring copy_to for copying data instead of duplicating the payload for both fields on client side. We realized that ES doesn't support copy_to for copying data from an object to nested type. It has been discussed here with the explanation -

Because there can be multiple nested objects - each one is indexed as a separate document. Which one do you copy the object field to?

Looking at the code, it seems like it might be possible to identify the document each nested field copies data to based on the parsing context of original source. For ex. for a mapping like below while parsing value of 'Name' field in 'Reviews, it should be possible to use the source doc in the bulk API request to uniquely identify & create a separate lucene document-

"mappings": {
		"docs": {
			"_all": {
				"enabled": false
			},
			"properties": {
				"Key": {
					"type": "string",
					"store": false,
					"index": "analyzed",
					"doc_values": false					
				},
                         "Reviews": {
					"type": "object",
					"properties": {
						"Name" : {
							"type": "string",
							"store": false,
							"index": "not_analyzed",
							"doc_values": false,
							"copy_to": "Posts.Name"
						}
					}
				},
                        "Posts": {
					"type": "nested",
					"properties": {
						"Name" : {
							"type": "string",
							"store": false,
							"index": "analyzed",
							"doc_values": false
						}
					}
				}
			}
		}
	}

(system) #2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.