NEST, number of transferred bytes when using compression

Elastic Stack Elasticsearch
1

Hi,

I use the NEST library to index documents. Request are made using a BulkAsync function. I enabled an http compression (gzip). I wonder whether it is possible to find out how many bytes were transferred when making a post request. I found a variable RequestBodyInBytes but it seems to be uncompressed body.

Thank you, Martin.

2

Hi Martin,

The short answer is that I don't believe there's an easy way to get this information today. You are correct that RequestBodyInBytes is the pre-buffered UTF8 JSON bytes useful for diagnostics but would not represent the compressed size. The compression happens lower in transport and uses the underlying HttpClient functionality.

If this is purely needed during development testing you could use a proxy such as Fiddler or a network tool such as WireShark to check the content-length header on requests. If you're on .NET 5+ then you could also look into the latest diagnostics telemetry provided by Microsoft. The sockets telemetry includes a bytes-sent counter you could track, although that would be for all sockets used by the application, not just NEST. I have a blog post that demonstrates those.

For NEST v8, I'm working on plans to expand the diagnostics and telemetry from the library. I will make sure this use case is captured and whether it can be included somewhere.

It would be great to learn more about your scenario to make sure its considered in any future design.

Steve

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.