Which technology to use for custom metrics for node.js?

Elastic Observability APM
,
1

I want to watch some metrics in my nodejs app. Those metrics are some values (int, double, ...). I see the APM Node.js API supports only transactions and spans. So, which API should I use to watch the metrics in Kibana? Should I send them by logs via LogStash? It's very important the values are tiny by size but very frequent by time.
The same question is for the Golang API. The documetation describes only working with system metrics, not with custom ones.

2

What metrics are you trying to monitor exactly? RAM, CPU, Disk I/O - use Metricbeat. Something really custom inside your Nodejs app - use the Nodejs agent, you can use its functionality to collect metrics that it doesn't take automatically. Here are the metrics it takes by default in addition to transactions and spans.

If you want to say "starting at this line, how long does it take to get to this line", try a custom span.

3

Emanuil, thanks for your response. I say about variables in my code, which I want to watch. It may be anything. They change and I need to see how they change on a graph. Is it possible? Which API should I use for it?

4

That part is pretty hard - APM doesn't allow you to define your own graphs. You could use a custom label to show the variable's value for each transaction. You'd need the setLabel function in the node.js agent. The data is searchable (both field name and whatever the value is) via the search bar in the APM UI. If your data is more complicated, you can attach an entire JSON object to each transaction via setCustomContext. This one isn't searchable though.

I'm not aware of other options to see the value of a variable for a given transaction with APM and Kibana. There are debuggers you can attach to running processes to watch vars, but that's unrelated to Elastic's tools, and I've not personally done it for node.js.

5

The agents do have an internal API through which metrics may be published (they're used to publish the built-in metrics), however not all of the agents currently expose it. The Node.js and Go agents do:

The Go agent also provides integrations with the rcrowley/go-metrics (apmgometrics) and Prometheus APIs (apmprometheus). We currently only support publishing custom counter/gauge metrics, and not distributions/histograms.

Metrics will be stored in apm-<version>-metrics-* indices. They won't automatically show up in graphs anywhere at the moment (that may change later). You can visualise them using standard Kibana functionality, such as Lens and TSVB.

1 Like
6

Thank you guys for your help. I tried to use logs via LogStash to send a variable:

        request.post(
            'http://192.168.1.190:8090',
            {
                json:
                {
                    sender: 'rv-core',
                    timestamp: new Date().toISOString().replace(/T/, ' ').replace(/\..+/, ''),
                    fps: val
                }
            },
            function (error, response, body) {
                if (error) {
                    console.log("Error in post request")
                }
            }
        );

Here I send FPS values of a camera. It works, I build a graph, but I'm concerned about logs are not designed for it. And I cannot send many values in one request to lower traffic. Is it a right way to visualise a data of a variable?

I'm not sure if I undestood you... Will I be able to build a graph (line or gistogram) using API you described? What you mean "automatically"? I can write some SQL to visualise the data, it's not a problem. Or I can build only some types of graphs?

7

Here I send FPS values of a camera. It works, I build a graph, but I'm concerned about logs are not designed for it. And I cannot send many values in one request to lower traffic. Is it a right way to visualise a data of a variable?

This looks fine to me. The Elasticsearch documents will have a similar structure to the metrics that Elastic APM agents produce.

You could also index documents directly into Elasticsearch, and use the bulk API to reduce the number of requests.

I'm not sure if I undestood you... Will I be able to build a graph (line or gistogram) using API you described? What you mean "automatically"? I can write some SQL to visualise the data, it's not a problem. Or I can build only some types of graphs?

I just meant that there are no pre-built dashboards where your custom metrics will show up. You will have to create your own dashboard/visualisation. It sounds like you already intended to do that.

8

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.