NodeJS APM does not report postgres queries

Elastic Observability APM
1

I am an elastic newbie and slowly getting things setup. I have the ELK setup working through Bitnami (https://bitnami.com/stack/elk) v7.4.2. I then externally setup the APM server to access the elasticsearch server at localhost:9200. I am trying to instrument my NodeJS app with Elastic APM for performance monitoring.

I included the package and basic config at the top of myindex.js file like so:

var apm = require('elastic-apm-node').start({
    // add configuration options here
  })

And I start the APM server with command line config as follows:

apm-server.exe -e -E output.elasticsearch.hosts=localhost:9200 -E apm-server.host=localhost:8200

The APM server was configured with default settings:

apm-server setup --index-management
apm-server setup --pipelines

Upon starting the server, I see messages of a successful connection with elasticsearch and upon starting the nodejs process I see a successful connection.

The only error I see in the APM-server command line output repeated sometimes is:

2019-11-21T12:55:15.220-0800 ERROR [request] middleware/log_middleware.go:74 forbidden request {"request_id": "8dfaf986-c1f2-4c43-a2f2-f579781b911b", "method": "GET", "URL": "/config/v1/agents?service.name=wsdot_evse_sim_manager&service.environment=development", "content_length": 0, "remote_address": "127.0.0.1", "user-agent": "elasticapm-node/3.2.0 elastic-apm-http-client/9.2.1 node/10.5.0", "response_code": 403, "error": "forbidden request: endpoint is disabled"}

But I believe that is due to Kibana endpoint not being specified. However, the issue of this post is that while I see two indexes in my elasticsearch instance with names: apm-7.4.2-onboarding-2019.11.21 and apm-7.4.2-metric-2019.11.21. The former only has one document and the later and app metrics as expected.

The other repeated message in command line output of AP server is:

2019-11-21T13:00:24.817-0800 INFO [request] middleware/log_middleware.go:76 request accepted {"request_id": "23ee9cc1-3291-4bb5-9886-9a66186a81ba", "method": "POST", "URL": "/intake/v2/events", "content_length": -1, "remote_address": "127.0.0.1", "user-agent": "elasticapm-node/3.2.0 elastic-apm-http-client/9.2.1 node/10.5.0", "response_code": 202}

And this results in Kibana showing this JSON.

I do not, however, see any information about the Postgres queries in my index.js code.

I have the following postgres related code in my index.js:

const pg = require('pg');
const connectionString = `postgres://${process.env.MAIN_USER}:${process.env.MAIN_PWD}@${process.env.MAIN_HOST}:${process.env.MAIN_PORT}/${process.env.MAIN_DB}`;
const pgClient = new pg.Client(connectionString);
pgClient.connect();
const query = pgClient.query('LISTEN new_order')
pgClient.query("update analysis_record set status = 'queued' where analysis_id = " + a_id, (err, res) => {
   console.log(err, res)
   console.log("Status updated to queued");
   // pgClient.end()
});

The queries are executed successfully but nothing is captured in the apm-7.4.2-metric-2019.11.21 index.

Am I missing some configuration in the APM server/agent? Or in my elasticsearch or Kibana? Maybe there is a particular way to use pg so that APM successfully captures it? Another possibility is the query is so fast that APM misses it? An example would be helpful.

I am using OSS APM 7.4.2, and NodeJS: v10.5.0 with elastic-apm-node@3.2.0 on Windows 10 with PostgreSQL 12.

Thanks

2

Hi @chintanp, welcome to the forum!

The agent will only trace database queries if they're made within the context of a sampled transaction. Typically people use the agent with web applications, measuring the latency of requests, with details of operations such as database queries in those requests.

As yours is not a web application, you'll need to add some more instrumentation. Specifically, you'll need to start a transaction using apm.startTransaction before making the query, and end the transaction after the query has completed.

1 Like
3

@axw thank you for your response. Is this true for Java applications too ? As in ...does the default instrumentation only work for web apps ?

Thanks

4

https://www.elastic.co/guide/en/apm/agent/java/current/supported-technologies-details.html has details on what is supported for Java.

There is incubating support for JMS, which will create transactions when using a MessageListener. There is also support for some schedulers, such as Quartz. Otherwise, the Java agent also has the trace_methods configuration to dynamically instrument methods matching a specified pattern, creating transactions or spans.

5

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.