How do I send a custom metric from within my code?

Elastic Observability Metrics
1

I am in the process of understanding the Elastic product suite, and I don't fully understand 'Metrics'. I know I can send structured log events to Elasticsearch, and I can search and filter on specific fields that I added to the log events using Kibana. I can also build a visualization using those fields and their values.

But we don't retain the logs, and there is no such concept of sampling I could find. So log events are not the same as metrics, although log events can contain metric-like data (e.g. duration of a request: "GET /endpoint completed with status 200 in 345 ms").

When not using a regular log event, how do I use Elastic 'Metrics' to instrument my code and send "RequestDuration": 345, or "RequestSuccessful": +1?

2

You can use the official Elasticsearch clients to send metrics. See more at https://www.elastic.co/guide/en/elasticsearch/client/index.html

But as I see "request duration", maybe you might find APM useful?
See more at https://www.elastic.co/apm and https://www.elastic.co/guide/en/apm/get-started/current/index.html

Let us know!

3

I understand I can use the Elasticsearch client to send anything to Elasticsearch. I am using Serilog, a logging library for .NET which has a plugin to send structured logs to Elasticsearch.

According to your own learning materials on elastic.co, there is a difference between logs and metrics (and traces). They are different 'pillars of observability'. And Elastic has a product called 'Metrics'.

My questions are:

  • What makes something a metric?
  • How does an index that contains log events differ from an index that contains metrics?
  • How do I make sure that when I use an Elasticsearch client (indirectly via my logging library) to send a metric, it is persisted/handled as a metric event, and not as a log event?

PS. I have used Datadog before. Here is their relevant documentation for .NET developers: https://github.com/DataDog/dogstatsd-csharp-client.

1 Like
4

Elasticsearch is a distributed search engine and data store. It is also capable of generating calculations or extract statistics from the data we store into it.

For Elasticsearch metrics and logs are just documents with different structure and contents.

Metric documents usually contain synthetic stats about what happened in a transaction, or a short period of time.
Usually numeric values and statuses, counters, values.
They are usually periodic. They have a timestamp and metadata to know from which they originated.

Logs documents usually have a structured or unstructured message you've collected from an application or a service, plus a timestamp and metadata to know from which they originated.
Usually there's a "message" field which contains the key information about the log line.

The difference between them are the mappings of the indices and the contents.

The default metric dashboards point to indices named metricbeat because Metricbeat is the tool we usually suggest to collect and send metrics.

The default log dashboard and UI point to indices named filebeat or logstash, but this is configurable.

The advantage is you can use Elasticsearch and the other Elastic products or clients (or even non-elastic tools or libraries) to build your solution.

I hope you've answered your questions.

Elastic has several videos and blogposts about metrics, logs. E.g. https://www.elastic.co/webinars/agumenting-logs-and-metrics-with-apm

5

Thank you, it helps!

1 Like
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.