NEST 6 - Serialization of enums as strings in terms query

[jlangmaid (Jonathan Langmaid)]

Hi there,

I've been trying to update to ES6 and NEST 6 and running into issues with NEST serializing of search requests - specifically serializing Terms queries where the underlying C# type is an enum. This is targeting NEST version 6.2.0

I've got a Status enum mapped in my index as a Keyword, and correctly being stored in its string representation by using NEST.JsonNetSerializer and setting the contract json converter

The issue comes when trying to search based on this Status enum. When I try to use a Terms query to specify multiple values, these values are being serialized as integers in the request and causing the search to find no results due to the type mismatch.

Interestingly the enum is serialized correctly as a string in a Term query, so I'm theorizing that the StringEnumConverter is being ignored in a scenario where it's having to serialize a collection of enums rather than a single enum.

Lets show it a little more clearly in code. Here's the enum and the (simplified) model used to define the index:

public enum CampaignStatus
{
    Active = 0,
    Sold = 1,
    Withdrawn = 2
}

public class SalesCampaignSearchModel
{
    [Keyword]
    public Guid Id { get; set; }

    [Keyword(DocValues = true)]
    public CampaignStatus CampaignStatus { get; set; }
}

Here's a snippet of constructing the settings for the ElasticClient:

var pool = new SingleNodeConnectionPool(new Uri(nodeUri));
var connectionSettings = new ConnectionSettings(pool, (builtin, serializerSettings) => 
        new JsonNetSerializer(builtin,
            serializerSettings,
            contractJsonConverters: new JsonConverter[]{new StringEnumConverter()}
        )
    )
    .EnableHttpCompression();

Here's the Term query that correctly returns results:

var singleTermFilterQuery = new SearchDescriptor<SalesCampaignSearchModel>()
    .Query(x => x.Term(y => y.Field(z => z.CampaignStatus).Value(CampaignStatus.Active)));

Generating the request:

{
  "query": {
    "term": {
      "campaignStatus": {
        "value": "Active"
      }
    }
  }
}

Here's the Terms query that does not return results:

var termsFilterQuery = new SearchDescriptor<SalesCampaignSearchModel>()
    .Query(x => x.Terms(y => y.Field(z => z.CampaignStatus).Terms(CampaignStatus.Active, CampaignStatus.Sold)));

Generating the request:

{
  "query": {
    "terms": {
      "campaignStatus": [
        0,
        1
      ]
    }
  }
}

So far I've had a pretty good poke around at the options being presented by the JsonNetSerializer, tried a bunch of the available attributes (NEST.StringEnumAttribute, [JsonConverter(typeof(StringEnumConverter))] rather than using the global one on the client, having an explicit filter object with ItemConverterType set on the collection of CampaignStatuses, etc.) and the only thing that has had any success was a very brute-force .ToString() every time I need to query on an enum.

These are toy examples from a reasonably extensive codebase that I'm trying to migrate across to NEST 6, so what I'm wanting is to be able to specify global configuration somewhere rather than multiple developer teams needing to be mindful of this kind of eccentricity.

So yeah... I've been looking at this for a couple of days now. Good chances there's something silly I've missed. Otherwise I'm wondering if I need to be providing some JsonConverter with a contract that would match to an arbitrary collection of enums, and whether NEST and their tweaked Json.NET serializer should just be doing that kind of recursive resolution out of the box already.

Any help would be greatly appreciated, as I'm going a bit crazy with this one.