We need to upgrade from NEST to the new API and are DESPARATELY looking for courses (paid or unpaid) to guide us through this process.
Can anyone help? Thanks
Hello @xef
I am not sure if there are specific courses for this topic but see below few elastic documentation if it can be helpful :
Thanks!!
Thanks for the links. But they are so basic they cover nothing apart from a couple of really simple issues.
It seems we have to move our Elasticsearch to Azure AI Search as we do not want to go through this hassle again.
Hi @xef ,
I'm the current maintainer of the .NET client.
Elasticsearch has an API surface of ~400 endpoints and ~3000 types. We have to focus on the most important parts for our generic migration guides, but I'm more than happy to help you with the migration, if you could name a few concrete examples.
Thanks for your response.
The first thing we need to figure out is how to convert such a filter to the new API where there are infact 2 indicies involved:
filters.Add(fq => fq.Term(t => t.Field(f => f.SecondaryLocalityId).Value(locationParams[2])) || fq.GeoShape(g => g.Field("locationShape").Relation(GeoShapeRelation.Within).IndexedShape(f => f.Id(searchCriteria.spLocationId).Index(indexName).Path($"geometry{buffer}"))));
also an ingestion xample for something like:
ElasticSearchConfig.GetClient().Ingest.PutPipeline("agent-pipeline", p => p
.Processors(ps => ps
.Circle(s => s
.Field(t => t.OperationBoundary).ErrorDistance(agent.OperationRadius * 1.02 * 0.01).ShapeType(ShapeType.GeoShape)
)
));
agent.OperationBoundary = $"CIRCLE({agent.Longitude} {agent.Latitude} {agent.OperationRadius})";
Thank you.
@xef I attached a full demo for both cases. In the first case, || and && for queries can not be used inline anymore and require manual .Should() or .Must() blocks. The second case with the ingest pipeline has not really changed much.
// Demo for https://discuss.elastic.co/t/courses-on-elastic-clients-elasticsearch/386125/5
// Translates two NEST-era snippets to the Elastic.Clients.Elasticsearch v9 surface.
using System.Text.Json.Serialization;
using Elastic.Clients.Elasticsearch;
using Elastic.Clients.Elasticsearch.Ingest;
using Elastic.Clients.Elasticsearch.QueryDsl;
namespace Playground;
public sealed class Location
{
[JsonPropertyName("secondaryLocalityId")]
public string? SecondaryLocalityId { get; init; }
[JsonPropertyName("locationShape")]
public object? LocationShape { get; init; }
}
public sealed class Agent
{
[JsonPropertyName("longitude")]
public double Longitude { get; init; }
[JsonPropertyName("latitude")]
public double Latitude { get; init; }
[JsonPropertyName("operationRadius")]
public double OperationRadius { get; init; }
[JsonPropertyName("operationBoundary")]
public string? OperationBoundary { get; set; }
}
public sealed record SearchCriteria(string SpLocationId);
public static class DiscussDemo
{
public static async Task RunAsync(ElasticsearchClient client)
{
// ---------- Inputs from the original snippet ----------
var locationParams = new[] { "loc-A", "loc-B", "loc-C" };
var searchCriteria = new SearchCriteria(SpLocationId: "shape-doc-42");
var indexName = "shape-index";
var buffer = "_500m";
// ---------- Snippet 1: Bool filter (Term OR GeoShape against another index) ----------
// NEST original:
// filters.Add(fq =>
// fq.Term(t => t.Field(f => f.SecondaryLocalityId).Value(locationParams[2]))
// || fq.GeoShape(g => g.Field("locationShape")
// .Relation(GeoShapeRelation.Within)
// .IndexedShape(f => f.Id(searchCriteria.spLocationId)
// .Index(indexName)
// .Path($"geometry{buffer}"))));
//
// In the new client there is no `||` operator on QueryDescriptor; OR-ing two leaves
// inside a Filter clause is expressed as a nested Bool with `Should`.
var searchResponse = await client.SearchAsync<Location>(s => s
.Indices("locations")
.Query(q => q
.Bool(b => b
.Filter(f => f
.Bool(inner => inner
.Should(
should => should
.Term(t => t
.Field(d => d.SecondaryLocalityId)
.Value(locationParams[2])),
should => should
.GeoShape(g => g
.Field("locationShape")
.Shape(shape => shape
.Relation(GeoShapeRelation.Within)
.IndexedShape(idx => idx
.Id(searchCriteria.SpLocationId)
.Index(indexName)
.Path($"geometry{buffer}"))))))))));
Console.WriteLine($"[snippet 1] search status: {searchResponse.ApiCallDetails.HttpStatusCode}, hits: {searchResponse.Total}");
// ---------- Snippet 2: Ingest pipeline with Circle processor ----------
// NEST original:
// Ingest.PutPipeline("agent-pipeline", p => p
// .Processors(ps => ps
// .Circle(s => s
// .Field(t => t.OperationBoundary)
// .ErrorDistance(agent.OperationRadius * 1.02 * 0.01)
// .ShapeType(ShapeType.GeoShape))));
var agent = new Agent { Longitude = 13.4, Latitude = 52.5, OperationRadius = 250 };
agent.OperationBoundary = $"CIRCLE({agent.Longitude} {agent.Latitude} {agent.OperationRadius})";
var putPipeline = await client.Ingest.PutPipelineAsync("agent-pipeline", p => p
.Processors(ps => ps
.Circle<Agent>(c => c
.Field(t => t.OperationBoundary)
.ErrorDistance(agent.OperationRadius * 1.02 * 0.01)
.ShapeType(ShapeType.GeoShape))));
Console.WriteLine($"[snippet 2] put pipeline acknowledged: {putPipeline.Acknowledged}");
}
}
Thank you very much.
As we might need assistance quite a bit in upgrading and your time is obviously very limited, are there any consultnats that we can hire to assist us in this?
Also, when indexing documents, does the old method of defining index mappings such as 
internal static class ElasticSearchIndexConfigServices
{
internal static CreateIndexDescriptor IndexDescriptorPosts(string indexName)
{
return new CreateIndexDescriptor(indexName)
.Settings(s => s
.NumberOfShards(2)
.NumberOfReplicas(1)
.Analysis(a => a
.Normalizers(n => n.Custom("lowercase", cn => cn.Filters("lowercase")))
.Analyzers(aa => aa
.Standard("standard_english", sa => sa.StopWords("_english_"))
)
)
)
.Map<ServiceItem>(map => map
.AutoMap()
.Properties(ps => ps
.Keyword(s => s.Name(c => c.Id))
.Keyword(d => d.Name(c => c.PromoterId))
.Keyword(s => s.Name(c => c.Username).Normalizer("lowercase"))
.Text(d => d.Name(c => c.PromoterName))
.Text(d => d.Name(c => c.PromoterNameInt))
etc. etc.
Thanks again.
Also, as i understand it, we can still use NEST with the latest version of elasticsearch (version 9) in addition to the new .NET API. Is this correct?
P.S. I would have expected the new version would make things simpler, but it seems things have become even more complicated.
// ---------- Snippet 3 (post #7): typed CreateIndex with settings + analysis + mappings ----------
// NEST original:
// new CreateIndexDescriptor(indexName)
// .Settings(s => s.NumberOfShards(2).NumberOfReplicas(1)
// .Analysis(a => a
// .Normalizers(n => n.Custom("lowercase", cn => cn.Filters("lowercase")))
// .Analyzers(aa => aa.Standard("standard_english", sa => sa.StopWords("_english_")))))
// .Map<ServiceItem>(map => map
// .AutoMap()
// .Properties(ps => ps
// .Keyword(s => s.Name(c => c.Id))
// .Keyword(d => d.Name(c => c.PromoterId))
// .Keyword(s => s.Name(c => c.Username).Normalizer("lowercase"))
// .Text(d => d.Name(c => c.PromoterName))
// .Text(d => d.Name(c => c.PromoterNameInt))));
//
// Notable differences in v9:
// * `.AutoMap()` no longer exists — properties must be enumerated explicitly (or you can build a
// `TypeMapping` from a JSON file/template). Inferred mapping based on CLR types is gone.
// * `.Properties(ps => ps.Keyword(s => s.Name(c => c.Id)))` becomes the more concise
// `.Properties(p => p.Keyword(c => c.Id))` — the property descriptor lambda is passed the
// field expression directly (no separate `.Name(...)` call).
// * NEST's `Filters("lowercase")` is now `Filter("lowercase")` on `CustomNormalizerDescriptor`.
// * NEST's `StopWords("_english_")` is now `Stopwords(StopWordLanguage.English)` (the enum value
// serializes to `_english_`).
var serviceIndexName = "service-items";
if ((await client.Indices.ExistsAsync(serviceIndexName)).Exists)
await client.Indices.DeleteAsync(serviceIndexName);
var createIndex = await client.Indices.CreateAsync<ServiceItem>(serviceIndexName, c => c
.Settings(s => s
.NumberOfShards(2)
.NumberOfReplicas(1)
.Analysis(a => a
.Normalizers(n => n
.Custom("lowercase", cn => cn.Filter("lowercase")))
.Analyzers(aa => aa
.Standard("standard_english", sa => sa.Stopwords(StopWordLanguage.English)))))
.Mappings(m => m
.Properties(p => p
.Keyword(c => c.Id)
.Keyword(c => c.PromoterId)
.Keyword(c => c.Username, k => k.Normalizer("lowercase"))
.Text(c => c.PromoterName)
.Text(c => c.PromoterNameInt))));
Also, as i understand it, we can still use NEST with the latest version of elasticsearch (version 9) in addition to the new .NET API. Is this correct?
It might technically work, but I can't recommend it, nor guarantee anything. This setup will brick the second ES introduces breaking changes in any of the used APIs.
NEST is compatible to ES 7 and ES 8 (if compatibility mode is enabled). Everything else is not officially supported from our side.
I would have expected the new version would make things simpler, but it seems things have become even more complicated.
The linked blog post here explains the reasons for switching from a fully hand-crafted NEST client to the largely code generated new client:
There is some syntactic sugar missing from the new client, but overall the design is more consistent and IMO also easier once you got used to it. The structure of the types and descriptor methods maps nearly 1:1 to the underlaying JSON REST API types.
As we might need assistance quite a bit in upgrading and your time is obviously very limited, are there any consultnats that we can hire to assist us in this?
Please have a look here: Consulting Services for the Elastic Stack | Elastic Consulting
Thanks. Are partial updates supported?
var o = new { PaidTill = DateTime.UtcNow.AddMonths(Utils.ExtendListingLife) };
var partialUpdateResponse = await client.UpdateAsync<PropertySearchResult, object>(id, u => u.Index($"properties-{country.ToLower()}").Doc(o).RetryOnConflict(3));
And multiplesearches
var ms = new MultiSearchDescriptor();
1-ms.Search("featured", u => u.Index($"properties-{searchCriteria.spCountry.ToLower()}")...
2-ms.Search<People> etc
var result = await ElasticSearchConfig.GetClient().MultiSearchAsync(ms);
where ms are multiple searches
If they differ from the above would appreciate an example.
Thanks
Just in case anyone is looking for Partial Update I think this is the new way of doing it:
await client.UpdateAsync<MyDoc, object>("my-index", id, u => u .Doc(new { Status = "active", UpdatedAt = DateTime.UtcNow }) );
Please confirm if this is correct and would still aprreciate your response on MultipleSearches.
Thaks
Hi @xef — answering the open questions one by one.
1) Partial update (re: post #10)
Yes, your snippet is correct. The UpdateAsync<TDocument, TPartialDocument> overload taking (IndexName, Id, descriptor) is exactly the supported v9 shape. Using object as the partial-document type lets you pass an anonymous type for the Doc(...) value, which matches the NEST ergonomics:
var partialUpdateResponse = await client.UpdateAsync<PropertySearchResult, object>(
$"properties-{country.ToLower()}",
id,
u => u
.Doc(new { PaidTill = DateTime.UtcNow.AddMonths(12) })
.RetryOnConflict(3));
A few things to note vs. NEST:
.Index(...)is no longer a fluent call on the descriptor — the index name is the first positional argument ofUpdateAsync. (It is also still settable onUpdateRequest<TDoc, TPartial>if you build the request manually.).RetryOnConflict(...),.DocAsUpsert(...),.Upsert(...),.Script(...),.Refresh(...)etc. all still live on the descriptor.- If you want a strongly-typed partial doc instead of
object, declare a small POCO and substitute it for the second generic parameter — that helps when the partial shape is reused.
2) Multi-search (re: post #9)
The NEST-style MultiSearchDescriptor with a per-call .Search<T>(name, selector) API is gone in Elastic.Clients.Elasticsearch. Each sub-search is now a SearchRequestItem(MultisearchHeader, MultisearchBody), and you hand a collection of those to .Searches(...):
using Elastic.Clients.Elasticsearch.Core.MSearch;
using Elastic.Clients.Elasticsearch.QueryDsl;
var msearchResponse = await client.MultiSearchAsync<object>(ms => ms
.Searches(
new SearchRequestItem(
new MultisearchHeader { Indices = $"properties-{searchCriteria.spCountry.ToLower()}" },
new MultisearchBody
{
Size = 10,
Query = new MatchAllQuery() // or any typed Query, e.g. new BoolQuery { Filter = [...] }
}),
new SearchRequestItem(
new MultisearchHeader { Indices = "people" },
new MultisearchBody
{
Size = 5,
Query = new TermQuery("name.keyword", "alice")
})));
foreach (var response in msearchResponse.Responses)
{
// response is a MultiSearchResponseItem<object>; the result is either
// MultiSearchItem<object> (success) or MultiSearchError (failure).
}
Notes:
MultisearchBody.Querytakes a concreteElastic.Clients.Elasticsearch.QueryDsl.Query— i.e.new MatchAllQuery(),new TermQuery(field, value),new BoolQuery { Filter = [...] }, etc. All the same query classes you'd compose inside a normalSearchRequest.- Because sub-searches can return different document types, the simplest cross-type pattern is
MultiSearchAsync<object>and reading each item'sDocuments(you can deserialize them per item, or use typed POCOs if all items share one type). - The descriptor surface here is intentionally small — the heavy lifting is in
MultisearchBody, which has the same shape as a regular search request (From,Size,Sort,Aggregations,Source,PostFilter,Suggest, etc.).
3) "Where are all the geospatial classes? PointGeoShape etc."
Short answer: the typed geo-shape DOM from NEST (PointGeoShape, LineStringGeoShape, PolygonGeoShape, MultiPointGeoShape, MultiLineStringGeoShape, MultiPolygonGeoShape, CircleGeoShape, EnvelopeGeoShape, GeometryCollection, IGeoShape) does not exist in Elastic.Clients.Elasticsearch and is unlikely to come back as-is — the code-generated client only models what the Elasticsearch specification defines, and the spec describes geo-shape values as free-form GeoJSON/WKT rather than as a closed type hierarchy.
What the client does ship:
| Concept | v9 type |
|---|---|
geo_point mapping |
Mapping.GeoPointProperty |
geo_shape mapping |
Mapping.GeoShapeProperty |
A single geo_point value |
GeoLocation (lat/lon, geohash, array, WKT string) |
geo_shape / shape query body |
QueryDsl.GeoShapeFieldQuery — Shape is object? |
| Spatial relation enum | GeoShapeRelation |
geo_distance, geo_bounding_box, geo_polygon, geo_grid |
GeoDistanceQuery, GeoBoundingBoxQuery, GeoPolygonQuery, GeoGridQuery |
Aggregations (geo_bounds, geo_centroid, geohash_grid, geotile_grid, geohex_grid, geo_line) |
Aggregations.GeoBoundsAggregation etc. |
For the actual shape values (everything you used to express via new PointGeoShape(...), new PolygonGeoShape(...), …) the recommendation is:
Define concrete POCOs in your own code base that serialize to the GeoJSON shape Elasticsearch expects, and use them wherever you used to use NEST's geo-shape classes.
GeoShapeFieldQuery.Shapeis intentionally typed asobject?precisely so you can plug your own types in.
The minimal shape hierarchy is small and self-contained. A starting point that compiles against System.Text.Json is roughly this — drop it into your project and extend with whatever members you actually use:
using System.Text.Json.Serialization;
[JsonPolymorphic(TypeDiscriminatorPropertyName = "type")]
[JsonDerivedType(typeof(PointGeoShape), "Point")]
[JsonDerivedType(typeof(LineStringGeoShape), "LineString")]
[JsonDerivedType(typeof(PolygonGeoShape), "Polygon")]
[JsonDerivedType(typeof(MultiPointGeoShape), "MultiPoint")]
[JsonDerivedType(typeof(MultiLineStringGeoShape), "MultiLineString")]
[JsonDerivedType(typeof(MultiPolygonGeoShape), "MultiPolygon")]
[JsonDerivedType(typeof(GeometryCollection), "GeometryCollection")]
public abstract class GeoShape { }
public sealed class PointGeoShape : GeoShape
{
public PointGeoShape() { }
public PointGeoShape(double longitude, double latitude) => Coordinates = [longitude, latitude];
[JsonPropertyName("coordinates")]
public double[] Coordinates { get; set; } = [];
}
public sealed class LineStringGeoShape : GeoShape
{
[JsonPropertyName("coordinates")]
public double[][] Coordinates { get; set; } = [];
}
public sealed class PolygonGeoShape : GeoShape
{
// Outer ring first, optional holes after.
[JsonPropertyName("coordinates")]
public double[][][] Coordinates { get; set; } = [];
}
public sealed class MultiPointGeoShape : GeoShape { [JsonPropertyName("coordinates")] public double[][] Coordinates { get; set; } = []; }
public sealed class MultiLineStringGeoShape : GeoShape { [JsonPropertyName("coordinates")] public double[][][] Coordinates { get; set; } = []; }
public sealed class MultiPolygonGeoShape : GeoShape { [JsonPropertyName("coordinates")] public double[][][][] Coordinates { get; set; } = []; }
public sealed class GeometryCollection : GeoShape
{
[JsonPropertyName("geometries")]
public GeoShape[] Geometries { get; set; } = [];
}
// Circle is not GeoJSON-standard — Elasticsearch accepts it as a WKT string,
// or via the geo_shape API with type=Circle plus a radius (only on the indexed
// document side, not in queries).
public sealed class CircleGeoShape : GeoShape
{
[JsonPropertyName("coordinates")]
public double[] Coordinates { get; set; } = [];
[JsonPropertyName("radius")]
public string Radius { get; set; } = ""; // e.g. "100m"
}
Usage is then identical to NEST:
var shape = new PolygonGeoShape
{
Coordinates =
[
[ [100.0, 0.0], [101.0, 0.0], [101.0, 1.0], [100.0, 1.0], [100.0, 0.0] ]
]
};
var resp = await client.SearchAsync<MyDoc>(s => s
.Indices("locations")
.Query(q => q
.GeoShape(g => g
.Field("locationShape")
.Shape(fq => fq
.Relation(GeoShapeRelation.Intersects)
.Shape(shape)))));
If you want to skip writing this from scratch, the simplest path is to copy the original NEST 7.17 sources verbatim and port the serialization layer to System.Text.Json:
src/Nest/QueryDsl/Geo/Shape/— the entire folder withGeoShapeBase,PointGeoShape,LineStringGeoShape,PolygonGeoShape,MultiPointGeoShape,MultiLineStringGeoShape,MultiPolygonGeoShape,CircleGeoShape,EnvelopeGeoShape,GeometryCollection,IGeoShape: https://github.com/elastic/elasticsearch-net/tree/7.17.5/src/Nest/QueryDsl/Geo/ShapeGeoCoordinatelives insrc/Nest/QueryDsl/Geo/GeoLocation.cson that same tag: https://github.com/elastic/elasticsearch-net/blob/7.17.5/src/Nest/QueryDsl/Geo/GeoLocation.cs
The data shapes (coordinates as nested double[] arrays, type as the discriminator, geometries for collections, radius for circles) are exactly what Elasticsearch expects on the wire, so the port is mechanical — strip the Utf8Json formatters and replace [DataMember(Name=...)] with [JsonPropertyName(...)] (or use the polymorphic attribute setup above to do the discrimination for you). The resulting types plug straight into GeoShapeFieldQuery.Shape and you keep the NEST call sites you already have.
For indexing, model your geo_shape field as GeoShape (the base type) on your document POCOs — System.Text.Json polymorphism handles the type discriminator on read and write, so round-tripping just works.
For the IndexedShape lookup (referencing a shape stored in another document) you already used in the earlier snippet, no DOM is needed — that lookup is Index/Id/Path and is fully supported.