Renaming an existing field in a nested object

[waynesmallman]

Hi!

I need to rename an existing field in a nested object:

"shared": {
	"type": "nested",
	"properties": {
		"user_id": {
			"type": "long"
		},
		"creation": {
			"type": "date",
			"format": "date_hour_minute_second"
		},
		"access": {
			"type": "keyword"
		}
	}
}

… and so far I have the following request:

PUT _ingest/pipeline/rename_field_shared_access
{
 "description" : "Rename the 'shared.access' field to: 'shared.access_type'",
 "processors" : [
  {
   "rename": {
    "field": "shared.access",
    "target_field": "shared.access_type"
   }
  }
 ]
}

I'm assuming the dot notation is correct?

Also, I would need to extend the nested object with additional fields:

"shared": {
	"type": "nested",
	"properties": {
		"user_id": {
			"type": "long"
		},
		"creation": {
			"type": "date",
			"format": "date_hour_minute_second"
		},
		"modification": {
			"type": "date",
			"format": "date_hour_minute_second"
		},
		"access_type": {
			"type": "keyword"
		},
		"resource_recipient_type": {
			"type": "keyword"
		},
		"resource_recipient_id": {
			"type": "long"
		}
	}
}

I'm 99.9% this is possible using Dynamic Mapping, but I need to check with the experts!

[xeraa]

I'm slightly confused — are we supposed to try it out for you or what is the specific question?

And yes, with dynamic mapping you can just add more fields to a document and they will be add with the default data types. But you could also extend the mapping manually if you prefer that.

[waynesmallman]

I need to know if the PUT request is correct, given the sample from the mapping I'm using.

What would it involve to extend the mapping?

[xeraa]

I need to know if the PUT request is correct, given the sample from the mapping I'm using.

Yes, but validate it for your own setup with:

PUT my-index/_doc/1?pipeline=rename_field_shared_access
{
  "shared": {
    "access": "foo"
  }
}

What would it involve to extend the mapping?

The docs have a complete example of updating a mapping after creating it . Which should look like:

PUT /my-index/_mapping
{
  "properties": {
    "shared": {
      "type": "nested",
      "properties": {
        "user_id": {
          "type": "long"
        },
        "creation": {
          "type": "date",
          "format": "date_hour_minute_second"
        },
        "modification": {
          "type": "date",
          "format": "date_hour_minute_second"
        },
        "access_type": {
          "type": "keyword"
        },
        "resource_recipient_type": {
          "type": "keyword"
        },
        "resource_recipient_id": {
          "type": "long"
        }
      }
    }
  }
}

[waynesmallman]

Excellent, but I have got two more questions:

1. Given that the instructions are in the Rename Processor, do I need to send an object payload in the PUT sample you've provided?
2. Is it possible to rename the field across an entire index? At a guess, remove the: "_doc/:id".

As for the mapping, I do have mappings (4 of them for an existing application), so I would need to use Dynamic Mapping to add the additional fields.

Thanks the response — much appreciated.

[xeraa]

• Given that the instructions are in the Rename Processor, do I need to send an object payload in the PUT sample you've provided?

No, there are multiple ways. An index could have a default ingest pipeline that is being applied. Or if data is being sent from a Beat you could specify that there. But you need need to define in one way or another which pipeline (if any) to use.

• Is it possible to rename the field across an entire index? At a guess, remove the: "_doc/:id".

No, that would require an _update_by_query or a full _reindex (both support ingest processors and scripts), but those are heavy operations since every change is a delete+ index operation in Lucene.