Updating nested objects

waynesmallman · May 30, 2017, 2:53pm

Hi, I've read through the Update API and read advice elsewhere, but it's still not clear to me how to update a specific nested object.

Here's the mappings and settings I'm using for the index, which I've also created an alias of "asset" for:

// PUT localhost:9200/asset_en_v1
{
"settings": {
	"analysis": {
		"char_filter": {
			"&_to_and": {
				"type": "mapping",
				"mappings": [ "&=> and "]
			}
		},
		"filter": {
			"asset_en_stopwords": {
				"type": "stop",
				"stopwords": [ "_english_" ]
			},
			"asset_en_stemmer": {
				"type": "stemmer",
				"name": "english"
			},
			"asset_en_shingle": {
				"type": "shingle",
				"max_shingle_size": 5,
				"min_shingle_size": 2,
				"output_unigrams": false,
				"output_unigrams_if_no_shingles": true
			}
		},
		"analyzer": {
			"asset_en_analyzer": {
				"type": "custom",
				"char_filter": [ "html_strip", "&_to_and" ],
				"tokenizer": "standard",
				"filter": [ "asset_en_stopwords", "asset_en_stemmer", "lowercase", "asset_en_shingle", "asciifolding" ]
			}
		}
	}
},
"mappings": {
	"note": {
		"_all": {
			"enabled": false
		},
		"properties": {
			"creation": {
				"type": "date",
				"format": "date_hour_minute_second"
			},
			"deleted": {
				"type": "integer"
			},
			"favourite": {
				"type": "integer"
			},
			"modification": {
				"type": "date",
				"format": "date_hour_minute_second"
			},
			"note": {
				"type": "text",
				"analyzer": "english",
				"fields": {
					"std": {
						"type": "text",
						"analyzer": "asset_en_analyzer",
						"fields": {
							"std": {
								"type": "text",
								"analyzer": "standard"
							}
						}
					}
				}
			},
			"title": {
				"type": "text",
				"analyzer": "english",
				"fields": {
					"std": {
						"type": "text",
						"analyzer": "asset_en_analyzer",
						"fields": {
							"std": {
								"type": "text",
								"analyzer": "standard"
							}
						}
					}
				}
			},
			"links": {
				"type": "nested",
				"include_in_parent": true,
				"properties": {
					"note_link_id": {
						"type": "long"
					},
					"user_id": {
						"type": "long"
					},
					"creation": {
						"type": "date",
						"format": "date_hour_minute_second"
					},
					"modification": {
						"type": "date",
						"format": "date_hour_minute_second"
					},
					"to_asset": {
						"type": "long"
					},
					"from_asset": {
						"type": "long"
					},
					"comment": {
						"type": "long",
						"analyzer": "english",
						"fields": {
							"std": {
								"type": "text",
								"analyzer": "asset_en_analyzer",
								"fields": {
									"std": {
										"type": "text",
										"analyzer": "standard"
									}
								}
							}
						}
					}
				}
			},
			"user_id": {
				"type": "long"
			}
		}
	}
}

What's relevant here is the "links" nested object, for example:

{
  "_index": "asset_en_v1",
  "_type": "note",
  "_id": "99",
  "_version": 2,
  "found": true,
  "_source": {
    "user_id": "11",
    "title": "Title",
    "note": "Note.",
    "creation": "2016-11-15T10:45:34",
    "modification": "2016-11-15T10:49:36",
    "links": [
      [
        {
          "note_link_id": "1",
          "user_id": "11",
          "creation": "2016-11-15T11:21:10",
          "modification": "2016-11-15T13:38:04",
          "to_asset": "100",
          "from_asset": "99",
          "comment": "Comment 1."
        },
        {
          "note_link_id": "2",
          "user_id": "11",
          "creation": "2016-11-15T13:37:04",
          "modification": "2016-11-15T13:37:27",
          "to_asset": "101",
          "from_asset": "99",
          "comment": "Comment 2."
        },
        {
          "note_link_id": "3",
          "user_id": "11",
          "creation": "2016-11-15T14:01:27",
          "modification": "2016-11-15T14:02:52",
          "to_asset": "102",
          "from_asset": "99",
          "comment": "Comment 3."
        }
      ]
    ],
    "favourite": 0,
    "deleted": "0"
  }
}

Over time, a user:

adds a link;
deletes a link;
updates the comment on a link.

I have an understanding of the adding and deleting, but the update is where things go wrong.

Any assistance would be much appreciated!

Tetrapack · May 31, 2017, 9:00am

Hello,

Firstly you can retrieve the list by getting the document. then, add / delete / update concerned link. Finally, send an update with the new updated list

exemple:

1°) comment of link with note_link_id 2 has been modified by user 11.

2°) perform a request like that :

GET asset_en_v1/note/_search
{
  "_source": "links",
  "query": {
    "bool": {
      "must": [
        {
          "term": {
            "user_id": {
              "value": "11"
            }
          }
        },
        {
          "nested": {
            "path": "links",
            "query": {
              "term": {
                "note_link_id": {
                  "value": "2"
                }
              }
            }
          }
        }
      ]
    }
  }
}

3°) Here you get the links list and the id of document to be updated. Update the comment of link 2

4°) make an update of the whole list:

POST asset_en_v1/note/99/_update
 {
  "doc" : {
    "links" : [ {...}, {...}, {...}]
  }
}

waynesmallman · May 31, 2017, 10:23am

Hi Emmanuel, so I don't need to use the script?

Tetrapack · May 31, 2017, 11:37am

Not necessarly, you can update a document with a script, or with a partial update.

-Using a script it could be something like this

POST asset_en_v1/note/99/_update
{
    "script" : {
        "inline": "ctx._source.links = params.newLinks",
        "lang": "painless",
        "params" : {
            "newLinks" : [{...}, {...}, ...]
        }
    }
}

depending on language you choose and complexity you want.

-Or with a partial update like my previous answer

I think that trying to make a fastidious script to only update a part of a nested list is purely useless.
Elasticsearch will reindex the whole document anyway.

The easiest way is to retrieved the list, update it locally, and send it entierly through a partial update (or scripted update)

waynesmallman · May 31, 2017, 12:01pm

I see.

Based on the examples (as stated in the opening question), I have the code:

$parameters = [
	'index' => "asset",
	'type' => $this->getAssetType( $from_note ),
	'id' => $from_note->note_id,
	'body' => [
		'script' => "ctx._source.links += newListItem",
		'params' => [
			'newListItem' => [
				'note_link_id' => $link_id,
				'user_id' => $this->flexi_auth->get_user_id(),
				'creation' => date('Y-m-d H:i:s'),
				'modification' => date('Y-m-d H:i:s'),
				'from_asset' => $from,
				'to_asset' => $to,
				'comment' => strip_tags($comment)
			]
		]
	]
];

But instead of updating the document, it appending some of the data to it, or — in previous versions of the code — overwrote the whole thing.

Emmanuel, thank you — I'll give the code a run tonight and report in with the results!

waynesmallman · May 31, 2017, 8:48pm

Hi Emmanuel, I'm having little success.

	$data = [
		'links' => [
			[
				'note_link_id' => $link_id,
				'user_id' => $this->flexi_auth->get_user_id(),
				'creation' => date('Y-m-d\TH:i:s'),
				'modification' => date('Y-m-d\TH:i:s'),
				'from_asset' => $from,
				'to_asset' => $to,
				'comment' => strip_tags( $comment )
			]
		]
	];

While that adds a link, if I attempt to add another, the first is overwritten.

Also, attempts at a partial update remove the fields not included in the update code.

Given this failure, I'll have to look at the script version for updates, assuming I'm able to get the adding of links working.

system · June 28, 2017, 8:48pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.