ElasticSearch - Nest - Mapping hits nested in lists and dictionaries

[h4tle (Michael Hatlehol)]

TLDR: How do I get the data I need to sit inside source like my timestamp and not inside a million lists and dictionaries?

I'm trying to create Data objects from this Nest query. The data comes from MetricBeats and ends up in a lot of nested lists and dictionaries inside my response.hits as you can see in the JSON response.

JSON Response

I would normally use this method to map my hits directly:

foreach (Hit<Data> item in dataResponse)
{
      Data d = item.Source;
}

Because the data is hidden so deep in the source this doesn't work. I can't use the Data object as my type in my search request which causes a lot of problems down the line.

I can't figure out whether the problem is my configuration of metricbeats, elasticsearch or my Nest query that causes the results to end up this way.

My search method

var response = await ElasticConnection.Instance.client.SearchAsync<dynamic>(s => s
            .Index("metricbeat-*")
                .Query(q => q
                    .Bool(b => b
                        .Should(sh => sh
                            .Match(c => c
                                .Field("host.network.in.bytes")
                                )
                            )
                        .Filter(f => f
                                .DateRange(dr => dr
                                .Field("@timestamp")
                                .GreaterThanOrEquals("now-5m")
                                )
                            )
                        )
                    )
                .Source(src => src
                    .Includes(i => i
                        .Field("host.network.in.bytes")
                        .Field("@timestamp")
                    )
                )
            );

This is what I'm doing at the moment to create a Data object with the data from my query. This can clearly be done in a different and more efficient way.

foreach (Hit<dynamic> item in dataResponse)
                {
                    Dictionary<string, dynamic> test = item.Source;
                    test.TryGetValue("host", out var host);
                    Dictionary<string, dynamic> test2 = host;
                    test2.TryGetValue("network", out var network);
                    Dictionary<string, dynamic> test3 = network;
                    test3.TryGetValue("in", out var input);
                    Dictionary<string, dynamic> test4 = input;
                    test4.TryGetValue("bytes", out var final);
                    test.TryGetValue("@timestamp", out dynamic time);
                    string test145 = time;
                    long test144 = final;
                    Data d = new Data
                    {
                        Bytes = test144,
                        Timestamp = test145
                    };
                    Console.WriteLine(d.Bytes);
                    data.Add(d);
                    Console.WriteLine(data);
                }

My Data class

public class Data
{
    public string Timestamp { get; set; }

    public long Bytes { get; set; }
}