Updating nested objects

Elastic Stack Elasticsearch
1

Hi, I've read through the Update API and read advice elsewhere, but it's still not clear to me how to update a specific nested object.

Here's the mappings and settings I'm using for the index, which I've also created an alias of "asset" for:

// PUT localhost:9200/asset_en_v1
{
"settings": {
	"analysis": {
		"char_filter": {
			"&_to_and": {
				"type": "mapping",
				"mappings": [ "&=> and "]
			}
		},
		"filter": {
			"asset_en_stopwords": {
				"type": "stop",
				"stopwords": [ "_english_" ]
			},
			"asset_en_stemmer": {
				"type": "stemmer",
				"name": "english"
			},
			"asset_en_shingle": {
				"type": "shingle",
				"max_shingle_size": 5,
				"min_shingle_size": 2,
				"output_unigrams": false,
				"output_unigrams_if_no_shingles": true
			}
		},
		"analyzer": {
			"asset_en_analyzer": {
				"type": "custom",
				"char_filter": [ "html_strip", "&_to_and" ],
				"tokenizer": "standard",
				"filter": [ "asset_en_stopwords", "asset_en_stemmer", "lowercase", "asset_en_shingle", "asciifolding" ]
			}
		}
	}
},
"mappings": {
	"note": {
		"_all": {
			"enabled": false
		},
		"properties": {
			"creation": {
				"type": "date",
				"format": "date_hour_minute_second"
			},
			"deleted": {
				"type": "integer"
			},
			"favourite": {
				"type": "integer"
			},
			"modification": {
				"type": "date",
				"format": "date_hour_minute_second"
			},
			"note": {
				"type": "text",
				"analyzer": "english",
				"fields": {
					"std": {
						"type": "text",
						"analyzer": "asset_en_analyzer",
						"fields": {
							"std": {
								"type": "text",
								"analyzer": "standard"
							}
						}
					}
				}
			},
			"title": {
				"type": "text",
				"analyzer": "english",
				"fields": {
					"std": {
						"type": "text",
						"analyzer": "asset_en_analyzer",
						"fields": {
							"std": {
								"type": "text",
								"analyzer": "standard"
							}
						}
					}
				}
			},
			"links": {
				"type": "nested",
				"include_in_parent": true,
				"properties": {
					"note_link_id": {
						"type": "long"
					},
					"user_id": {
						"type": "long"
					},
					"creation": {
						"type": "date",
						"format": "date_hour_minute_second"
					},
					"modification": {
						"type": "date",
						"format": "date_hour_minute_second"
					},
					"to_asset": {
						"type": "long"
					},
					"from_asset": {
						"type": "long"
					},
					"comment": {
						"type": "long",
						"analyzer": "english",
						"fields": {
							"std": {
								"type": "text",
								"analyzer": "asset_en_analyzer",
								"fields": {
									"std": {
										"type": "text",
										"analyzer": "standard"
									}
								}
							}
						}
					}
				}
			},
			"user_id": {
				"type": "long"
			}
		}
	}
}

What's relevant here is the "links" nested object, for example:

{
  "_index": "asset_en_v1",
  "_type": "note",
  "_id": "99",
  "_version": 2,
  "found": true,
  "_source": {
    "user_id": "11",
    "title": "Title",
    "note": "Note.",
    "creation": "2016-11-15T10:45:34",
    "modification": "2016-11-15T10:49:36",
    "links": [
      [
        {
          "note_link_id": "1",
          "user_id": "11",
          "creation": "2016-11-15T11:21:10",
          "modification": "2016-11-15T13:38:04",
          "to_asset": "100",
          "from_asset": "99",
          "comment": "Comment 1."
        },
        {
          "note_link_id": "2",
          "user_id": "11",
          "creation": "2016-11-15T13:37:04",
          "modification": "2016-11-15T13:37:27",
          "to_asset": "101",
          "from_asset": "99",
          "comment": "Comment 2."
        },
        {
          "note_link_id": "3",
          "user_id": "11",
          "creation": "2016-11-15T14:01:27",
          "modification": "2016-11-15T14:02:52",
          "to_asset": "102",
          "from_asset": "99",
          "comment": "Comment 3."
        }
      ]
    ],
    "favourite": 0,
    "deleted": "0"
  }
}

Over time, a user:

  1. adds a link;
  2. deletes a link;
  3. updates the comment on a link.

I have an understanding of the adding and deleting, but the update is where things go wrong.

Any assistance would be much appreciated!

2

Hello,

Firstly you can retrieve the list by getting the document. then, add / delete / update concerned link. Finally, send an update with the new updated list

exemple:

1°) comment of link with note_link_id 2 has been modified by user 11.

2°) perform a request like that :

GET asset_en_v1/note/_search
{
  "_source": "links",
  "query": {
    "bool": {
      "must": [
        {
          "term": {
            "user_id": {
              "value": "11"
            }
          }
        },
        {
          "nested": {
            "path": "links",
            "query": {
              "term": {
                "note_link_id": {
                  "value": "2"
                }
              }
            }
          }
        }
      ]
    }
  }
}

3°) Here you get the links list and the id of document to be updated. Update the comment of link 2

4°) make an update of the whole list:

POST asset_en_v1/note/99/_update
 {
  "doc" : {
    "links" : [ {...}, {...}, {...}]
  }
}
3

Hi Emmanuel, so I don't need to use the script?

4

Not necessarly, you can update a document with a script, or with a partial update.

-Using a script it could be something like this

POST asset_en_v1/note/99/_update
{
    "script" : {
        "inline": "ctx._source.links = params.newLinks",
        "lang": "painless",
        "params" : {
            "newLinks" : [{...}, {...}, ...]
        }
    }
}

depending on language you choose and complexity you want.

-Or with a partial update like my previous answer

I think that trying to make a fastidious script to only update a part of a nested list is purely useless.
Elasticsearch will reindex the whole document anyway.

The easiest way is to retrieved the list, update it locally, and send it entierly through a partial update (or scripted update)

5

I see.

Based on the examples (as stated in the opening question), I have the code:

$parameters = [
	'index' => "asset",
	'type' => $this->getAssetType( $from_note ),
	'id' => $from_note->note_id,
	'body' => [
		'script' => "ctx._source.links += newListItem",
		'params' => [
			'newListItem' => [
				'note_link_id' => $link_id,
				'user_id' => $this->flexi_auth->get_user_id(),
				'creation' => date('Y-m-d H:i:s'),
				'modification' => date('Y-m-d H:i:s'),
				'from_asset' => $from,
				'to_asset' => $to,
				'comment' => strip_tags($comment)
			]
		]
	]
];

But instead of updating the document, it appending some of the data to it, or — in previous versions of the code — overwrote the whole thing.

Emmanuel, thank you — I'll give the code a run tonight and report in with the results!

6

Hi Emmanuel, I'm having little success.

	$data = [
		'links' => [
			[
				'note_link_id' => $link_id,
				'user_id' => $this->flexi_auth->get_user_id(),
				'creation' => date('Y-m-d\TH:i:s'),
				'modification' => date('Y-m-d\TH:i:s'),
				'from_asset' => $from,
				'to_asset' => $to,
				'comment' => strip_tags( $comment )
			]
		]
	];

While that adds a link, if I attempt to add another, the first is overwritten.

Also, attempts at a partial update remove the fields not included in the update code.

Given this failure, I'll have to look at the script version for updates, assuming I'm able to get the adding of links working.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.