Best way to do a Bulk update with script

Elastic Search
1

I have been following a bulk update example to do a bulk update on my index however I get the following error:

Validation Failed: 1: can't provide both script and doc;

The below code is what is giving the error.

var bulkResponse = client.Bulk(b => b
    .Index("my_index")
    .UpdateMany(my_dictionary, (descriptor, dict) => descriptor
        .Id(dict.Key)
        .Script(s => s
            .Source("ctx._source.list.add(params.item)")
            .Params(p => p
                .Add("item", dict.Value)
            )
        )
    )
);

Would love some suggestions on how I can do a bulk update on multiple documents. The below works fine for single update on a document. I just need to apply it to bulk update.

var response = await _client.UpdateAsync<MyObject, MyObject>(
    index:"my_index",
    id:"my_id",
    u => u.Script(s => s
        .Source("ctx._source.list.add(params.item)")
        .Params(p => p
            .Add("item", myObject.Value))));
2

Hi @Matt_Shallow, Welcome to the Elastic Community.

  1. May i know which version of Elastic you're using?
  2. Also are you referring this javascript client ? It looks UpdateMany is not the part of it.
1 Like
3

Hi @ashishtiwari1993 . I am using the latest version of Elastic. Doing some exploration work.

I am using the latest available .NET Elastic client library. Documentation seems pretty lax on Bulk Api elastic docs for .NET. Even entries deleted :face_with_monocle: See below link.

Essentially, what I am trying to do is bulk update multiple documents adding a new item to a list I have in each of these documents. I am iterating through a dictionary whose Key = DocumentID and Value is the item to be added to the documents list.

Bulk UpdateMany seems exactly what I need but I am getting that error. So I need to somehow restructure the Query which I need help with :slightly_smiling_face:

4

Hi @Matt_Shallow,

you are hitting a very specific limitation of the UpdateMany API here.

The _bulk endpoint accepts either script or doc, but not both. In the client, we internally check which variant the user has provided. For this, we compare the value of script or doc to null.

In your specific case, TDocument is of type KeyValuePair<,> which is a value type. The null check yields incorrect results as value types are never null.

As a workaround, you could convert your KeyValuePair<,> to a reference type. This will cause a few extra allocations, but is currently the only way to make the bulk API work in the way you want:

var bulkResponse = await client.BulkAsync(b => b
	.Index("persons")
	.UpdateMany(my_dictionary.Select(x => new { x.Key, x.Value }), (descriptor, dict) => descriptor
		.Id(dict.Key)
		.Script(s => s
			.Source("ctx._source.list.add(params.item)")
			.Params(p => p
				.Add("item", dict.Value)
			)
		)
	)
);
1 Like
5

Thanks @flobernd for taking a look and solution. Let me give that a whirl and get back to you.

I will be doing some performance checks on different ways to store and then query the data in elastic so possibly my current approach might not be the way forward.

Thanks.

6

@flobernd This worked great and not really any noticeable performance issues. I appreciate you explaining clearly why I was getting that specific error and for providing a solution.

Thanks<
Matt