Async Background Search with .NET NEST Client

Elastic Stack Elasticsearch
1

How does one use the Elasticsearch Async Background search facility using the NEST Library.
We cannot find any documentation on the syntax.
Any help would be appreciated.

2

Hi Xef,

I assume with reference to NEST that you're using the Elasticsearch .NET library? Can you confirm which version of the client you're using? There has been a redesign of the client in v8.0, so it's useful to know.

3

Hi
Thanks for your response.
We are using the NEST client library 7.17.4
P.S. We are about to upgrade to 8, so if there are significant changes, may be you could show us the 8 version.
Thanks

4

Hi Xef,

I'm not a .NET expert so I've amended the title of your query to include reference to NEST so the client developers can pick it up or opine further on the answer or make any corrections if they know otherwise. But I think I've found a couple of useful things for you. Apologies in advance for the long answer!

Having a dig through the docs, I can't see a reference to async search for the NEST client. However, in the low level client, which is exposed in the NEST client via the .LowLevel property on NEST ElasticClient, it looks like available methods are exposed with synchronous and asynchronous versions with the convention of the *Async suffix. So in 7.17 I would guess you could use SearchAsync on the exposed low level client instead, which does look to be in the generated contract. Hopefully someone else will have another idea and be able to correct me if this isn't the case!

Looking forward to version 8 for when you upgrade, our docs do have an async search example here. Do take note of the installation warning regarding feature parity between 7 and 8 to check the release notes carefully to make sure all features you need for your application are present in 8 before upgrading.

Hope that helps!

5

Hi
Thanks for your response.
The link you have provided is the normal search just in async mode. It is not the background search facitlity which are 2 different things.

6

Fair enough. As I said I'm not a .NET expert, so we'll wait and see what others come back with. :smile: Hopefully the v8 references help with your future migration.

7

Hi
This is not a .NET feature. It is a functionality available in Elastic Search. All we want to know is how to use it with NEST. i.e. where in the docs is the NEST syntax.

The link to it using DSL in your docs is as follows:
Async search | Elasticsearch Guide [8.6] | Elastic.

8

Hi xef, As mentioned before I suggest waiting for the .NET client team or others with experience to come back as you're looking for an example using the NEST client since what's been provided before isn't what you're looking for. Responses on this forum are best effort.

With NEST now added to the title they should hopefully be able to pick it up as per the help guidelines in the .NET client docs. Best of luck!

9

Hi, @xef.

The NEST APIs mirror those available in Elasticsearch. The AsyncSearch APIs can be accessed from the client via the AsyncSearch property. We don't explicitly document this in the NEST docs, as the Elasticsearch docs cover async search usage.

You'd want something a bit like this as a starting point, though:

var searchId = string.Empty;

var asyncSearchSubmitResponse = await Client.AsyncSearch.SubmitAsync<Project>(a => a.Index("some-index").Query(q => q.MatchAll()));

if (asyncSearchSubmitResponse.IsValid)
{
	if (!asyncSearchSubmitResponse.IsRunning)
	{
		// handle results
		// asyncSearchSubmitResponse.Response.Hits ...
	}
	else
	{
		searchId = asyncSearchSubmitResponse.Id;
	}
}

// poll for results
if (!string.IsNullOrEmpty(searchId))
{
	while (true)
	{
		await Task.Delay(1000); // wait for 1 sec

		var asyncSearchStatusResponse = await Client.AsyncSearch.StatusAsync(searchId);

		if (!asyncSearchStatusResponse.IsRunning && asyncSearchStatusResponse.CompletionStatus == 200)
		{
			break;
		}
	}
}

var asyncSearchGetResponse = await Client.AsyncSearch.GetAsync<Project>(searchId);

if (asyncSearchGetResponse.IsValid)
{
	// handle results
	// asyncSearchGetResponse.Response.Hits ...
}

// cleanup
var deleteAsyncSearchResponse = await Client.AsyncSearch.DeleteAsync(searchId);

You initiate the search via the client.AsyncSearch.SubmitAsync method. Once you get a response, you need to check if the search was completed immediately or if it's still running asynchronously.

In the latter case, you can then poll for the status using the search ID until it is completed successfully. At that point, you can access the results and once you have those, delete the search on the server.

Does that help get you started?

Thanks, @carly.richmond, for stepping in as well.

1 Like
10

Yes. Thank you very much.

11

Is there by any chance the same background facility available not just for searching but for updating documents as well.
If not, what is the best option of updating a large number documents/requests in the background.
Thanks

12

For single document indexing, no async concept exists, as it should be pretty much immediate. There is the bulk API for multiple documents, and we have a BulkAll helper in the .NET client you can use to index many documents. That starts as a background Task and can be observed for status updates or waited upon for completion.

For updating, you can also use the Bulk endpoint.

13

Hi
Thanks for your response.
I was thinking more of updating documents in the background such as UpdateByQuery but in a background thread so that it does not slow down the current activities.
Thanks again.

14

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.