Possible to 'flatten' nested mapping with reindex?

Elastic Stack Elasticsearch
1

Hi... I have a nested mapping, which I'd like to reindex into a non-nested structure - i.e. each nested object + the 'parent' document becomes a new document. Is this possible by reindexing? I'd like to avoid reloading from the source ideally.

I've had a good play about and a google but haven't come across any winning combination yet...

Existing mapping is:

{
	"mappings" : {
		"properties" : {
		  "wsid" : {
			"type" : "keyword"
		  },
		  "date" : {
			"type" : "date",
			"index" : false
		  },
		  "md" : {
			"type" : "short"
		  },
		  "year" : {
			"type" : "short",
			"index" : false
		  },
		  "obs" : {
			"type" : "nested",
			"properties" : {
			  "hr" : {
				"type" : "byte"
			  },
			  "precipmm" : {
				"type" : "scaled_float",
				"index" : true,
				"scaling_factor" : 100.0
			  },
			  "tempc" : {
				"type" : "scaled_float",
				"index" : true,
				"scaling_factor" : 10.0
			  },
			  "snowcm" : {
				"type" : "scaled_float",
				"index" : true,
				"scaling_factor" : 100.0
			  },
			  "sunmins" : {
				"type" : "scaled_float",
				"index" : true,
				"scaling_factor" : 10.0
			  },
			  "cloudpct" : {
				"type" : "scaled_float",
				"index" : true,
				"scaling_factor" : 100.0
			  },
			  "windkph" : {
				"type" : "scaled_float",
				"index" : true,
				"scaling_factor" : 100.0
			  }
		  }
		}
	}
}

and what I'd like to get to:

{
	"mappings" : {
		"properties" : {
		  "wsid" : {
			"type" : "keyword"
		  },
		  "date" : {
			"type" : "date",
			"index" : false
		  },
		  "md" : {
			"type" : "short"
		  },
		  "year" : {
			"type" : "short",
			"index" : false
		  },
		  "hr" : {
			"type" : "byte"
		  },
		  "precipmm" : {
			"type" : "scaled_float",
			"index" : true,
			"scaling_factor" : 100.0
		  },
		  "tempc" : {
			"type" : "scaled_float",
			"index" : true,
			"scaling_factor" : 10.0
		  },
		  "snowcm" : {
			"type" : "scaled_float",
			"index" : true,
			"scaling_factor" : 100.0
		  },
		  "sunmins" : {
			"type" : "scaled_float",
			"index" : true,
			"scaling_factor" : 10.0
		  },
		  "cloudpct" : {
			"type" : "scaled_float",
			"index" : true,
			"scaling_factor" : 100.0
		  },
		  "windkph" : {
			"type" : "scaled_float",
			"index" : true,
			"scaling_factor" : 100.0
		  }
	  }
	}
}
2

Yes I think this is doable with the reindex API by using a script.

ex:

ctx._source.flat_field = ctx._source.nested_field.field;

But if you're using a nested mapping, the value will be an array of values.

https://www.elastic.co/guide/en/elasticsearch/reference/current/nested.html#_how_arrays_of_objects_are_flattened

Another solution would be to use logstash for this purpose.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.