How to read Different field names with same field value - Elasticsearch

Elastic Stack Elasticsearch
1

I have to separate index patterns that will have their own unique fields. For instance in

index1 it will contain a field property CorrelationId
index2 it will contain a field property UniqueCorrelationId

these two fields will share the same value

CorrelationId & UniqueCorrelationId: ffbdec21-5255-dee5-6b77-efaf6840481e.

I noticed that in order to be able to grab the values from ElasticSearch, I would need to have to correct field property name in order to be able to read it. If it is not that field property name, then it will return a null.

So in my model class I have a list of properties, i.e.

public class Fields
{
   ......

   [Text(Name = "CorrelationId")]
   public String CorrelationId {get; set;}

}

I am able to read the logs from both index patterns, but my issue I am coming across is if I leave the property name ("CorrelationId") in the model class and do not change it to "UniqueCorrelationId"). For the index pattern that contains the field property "UniqueCorrelationId", it will return null since it is not mapped correctly since the property is mapped to CorrelationId.

I was reading up on mapping in elasticsearch, and ran into auto mapping but it seems a bit confusing if this is truly the solution I am looking for. Where I can be able to map the fields without having to change the [Text(Name= "")] with NEST. Because some properties are different and it does not return the values from the logs in ElasticSearch unless the name of the mappings are matching exactly how it is shown in the index fields.

Is there a way to do this what I am trying to do?

2

Hi,

When indexing or searching to different indices, it's generally going to be easier to create separate POCO classes for each index. Those classes can model their properties to align closely with the fields in the index. Using the same model is challenging since the deserialiser has no way to know what to expect in the JSON.

If you only need to use your type for deserialisation, another option to consider is including two properties that align to both variations of field name. Then provide a computed property to access whichever has been set during deserialisation. Something like this:

public class Fields
{

    [Text(Name = "CorrelationId")]
    public string CorrelationId { get; set; }

    [Text(Name = "UniqueCorrelationId")]
    public string UniqueCorrelationId { get; set; }

    [Text(Name = "CorrelationId")]
    public string FinalCorrelationId => CorrelationId ?? UniqueCorrelationId;

}

Depending on your requirements, that may be sufficient.

Other options include registering a custom source serialisation and providing your own custom converters/formatters to handle the deserialisation. This is going to be more complex and harder to maintain.

The auto-mapping functionality is designed to provide a best guess for the mapping of C# properties to Elasticsearch fields, but it's generalised with common rules. If your requirements deviate from the generic case, you will need to handle those yourself, with attributes or fluent mapping syntax. In your example, you need the Text attribute to override the default name inferred for the property, since the default behaviour is to assume the name will be camel-cased in the JSON. You can read more about this and how to change that behaviour in our documentation.

Cheers,
Steve

1 Like
3

Thank you for your explanation. Before attempting to make a separate POCO class for each index, is there a way I can make it dynamic. As in the field names should come from configuration i.e. a json file

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.