Nest client. How to serialize date field to a custom format?

Elastic Stack Elasticsearch
1

Hi!
I use Nest 7.3 and a default serializer. And I have a problem with date serialization.
Suppose we have a model:

public class SearchEntity
{
    [Date(Format = "date_time_no_millis")]
    public DateTime Timestamp { get; set; }
}

I created index and a mapping of timestamp is OK.

Then I create an instance SearchEntity and try save it:

var entity = new SearchEntity { Timestamp = DateTime.Now };
client.Update(r => r
    .DocAsUpsert()
    .Doc(entity)
);

Here I get an error:

Type: mapper_parsing_exception Reason: "failed to parse field [timestamp] of type [date] in document with id '...'" CausedBy: "Type: illegal_argument_exception Reason: "failed to parse date field [2019-09-30T08:37:36.6356966+05:00] with format [date_time_no_millis]"

It doesn't matter, if I use date format date_time_no_millis, yyyy-MM-dd or any else, result the same.

What's wrong? How to set the field serialization format for DateTime property?

Thanks

2

NEST client always serializes DateTime, DateTimeOffset POCO properties and their nullable counterparts, as ISO8601 strings. the Format property of the Date attribute does not influence how the DateTime or DateTimeOffset instance is serialized.

I've been thinking about ways in which Format could be supported, but there's a few things to consider:

  • For simple date patterns where there is a direct mapping of the pattern understood by Elasticsearch to a pattern that would be understood by .NET date formatting, I think this is the easiest case to handle.
  • For named date patterns understood by Elasticsearch, format patterns would need to be implemented in the client to serialize according to the named pattern formats.
  • POCOs that use attributes to specify Format would allow the creation of a custom DateFormatter when generating a formatter for the POCO through reflection, but Format can also be specified through fluent mapping. In this latter case, creating a custom DateFormatter is tricky because there may be no reflection metadata available to inform the format to use. Mapping could be fetched from an index but I can see several problems with this approach which I believe would make it untenable. There is the option of specifying the format with both fluent mapping and an attribute but this doesn't feel like a great solution.

Open to ideas for how Format can be supported; more than happy to discuss on an issue on the .NET client GitHub repo :+1:

One way to address formatting in the example provided for now would be to format the DateTime in the POCO to the desired format, and send the formatted value to Elasticsearch

public class SearchEntity
{
	private DateTime _timestamp;
	
	[Nest.Date(Name = "timestamp", Format = "date_time_no_millis")]
	public string TimestampString
	{
		get => _timestamp.ToString("yyyy-MM-dd'T'HH:mm:ss");
		private set => _timestamp = DateTime.ParseExact(value, "yyyy-MM-dd'T'HH:mm:ss", CultureInfo.InvariantCulture);
	}
	
	[Ignore]
	public DateTime Timestamp
	{
		get => _timestamp;
		set => _timestamp = value;
	}
}

This would allow you to still work with DateTime in the POCO

    var client = new ElasticClient(settings);

    client.IndexDocument(new SearchEntity
	{
		Timestamp = new DateTime(2019,10,4,19,47,0)
	});

yields

POST http://localhost:9200/index/_doc
{"timestamp":"2019-10-04T19:47:00"}

Ensure a SearchEntity can be deserialized from the output

using (var stream = new MemoryStream(Encoding.UTF8.GetBytes("{\"timestamp\":\"2019-10-04T19:47:00\"}")))
{
    var doc = client.SourceSerializer.Deserialize<SearchEntity>(stream).Dump();
}
3

@forloop thank you for the answer. I'll open an issue on github and use this workaround :+1:

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.