Querying child objects of an indexed document (NEST)

Elastic Stack Elasticsearch
1

I think I'm missing something fundamental with querying via the .NET client. We're trying to search log events logged via Serilog, with a context object nested as a field (using the Serilog ForContext mechanism).

EventLog eventLog = ...;
log.ForContext("EventLog", eventLog, true).Write(eventLog.EventMessage);

This results in documents that look like:

{
  "_index": "logstash-2019.03.14",
  "_type": "logevent",
  "_id": "p_amfGkBaphWeJXGjjSW",
  "_version": 1,
  "_score": null,
  "_source": {
    "@timestamp": "2019-03-14T09:41:21.6924251-05:00",
    "level": "Information",
    "messageTemplate": "blah",
    "message": "blah",
    "fields": {
      "EventLog": {
        "_typeTag": "EventLog",
        "EventMessage": "blah",
        "EventId": 3112,
...

It seems like the only way we can query on properties of these context objects is by using the string form of the object path (not using a type-safe expression on the POCO):

client.Search<Dummy>(s => s
    .AllTypes()
    .Query(q => q
        .Match(t => t
            .Field(f => f.Message).Query("blah"); // returns some results

client.Search<Dummy>(s => s
    .AllTypes()
    .Query(q => q
        .Match(t => t
            .Field(f => f.Fields.EventLog.EventMessage).Query("blah"); // returns zero results

client.Search<Dummy>(s => s
    .AllTypes()
    .Query(q => q
        .Match(t => t
            .Field(f => new Field("fields.EventLog.EventMessage")).Query("blah"); // returns some results

What am I missing?

UPDATE: I enabled query logging in ES and saw that there's a case difference between the literal and expression queries. The literal query (with PascalCased properties) is seen as Pascal cased by ES and finds the documents, while the type-safe expression gets converted to camelCase ("fields.eventLog.eventMessage"), and does not match anything. Sure enough, if I query with a literal camelCased field path, I get nothing back also.

How do I tell NEST to use the right case sensitivity?

2

You can control field infererence globally with DefaultFieldNameInferrer delegate; it accepts the CLR property name as input, and allows you to transform however you see fit. If unspecified, NEST will camelcase property names.

var settings = new ConnectionSettings()
    .DefaultFieldNameInferrer(p => p);

var client = new ElasticClient(settings);

Looking at the _source in the example provided, you're in a little bit of a tricky situation regarding inferring field names from CLR POCO property names because you have a mix of camel case at the top level, and pascal case under "fields". You can apply attributes to POCO properties which NEST will use. For example

public class Dummy
{
    [PropertyName("fields")]
    public Fields Fields {get;set;}
}

public class Fields 
{
    [PropertyName("EventLog")]
    public EventLog EventLog {get;set;}
}

etc. You can use System.Runtime.Serialization.DataMemberAttribute instead of Nest.PropertyNameAttribute, if you prefer.

Then, lambda expressions will use these attribute values.

3

Thanks, Russ - that looks like exactly what I need. Serilog controls the "fields" part of the path written out to ES, and then logs the context objects and their properties with different case. Not sure why that is, but with these attributes we can take control over it, so it shouldn't matter.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.