# Not seeing APM data on elasticsearch/Kibana

**URL:** <https://discuss.elastic.co/t/not-seeing-apm-data-on-elasticsearch-kibana/187911>\
**Category:** APM\
**Tags:** nodejs\
**Created:** [June 27, 2019, 7:32pm UTC](https://discuss.elastic.co/t/not-seeing-apm-data-on-elasticsearch-kibana/187911 "2019-06-27T19:32:01Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Juan\_Stiza](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/juan_stiza/32/45092_2.png) [@Juan\_Stiza](https://discuss.elastic.co/u/Juan_Stiza)\
**Post date:** [June 27, 2019, 7:32pm UTC](https://discuss.elastic.co/t/not-seeing-apm-data-on-elasticsearch-kibana/187911/1 "2019-06-27T19:32:01Z")

</div>

**Kibana version** : 6.6.1

**Elasticsearch version** : 6.6.1

**APM Server version** : 6.6.1

**APM Agent language and version** : Node v2.11.6

**Browser version** : Chrome v75.0.3770.100

**Original install method (e.g. download page, yum, deb, from source, etc.) and version**:  
Cloud installation

**Steps to reproduce** :

1. Go to APM setup page
2. Copy snippets & set everything up with the client
3. Load APM config data
4. Launch APM

We have a cloud installation v6.6.1.  
We have an APM instance enabled, we have followed the steps on the setup page.  
We have set up the APM agent on two projects:

1. One legacy node project,
2. One Typescript, webpack compiled project.

We start both processes setting the config parameters as environment variables, and injecting the code by using `node -r elastic-apm-node/start app.js` method.

We have set the `trace` log level to see if anything looks ok.  
We get `agent configured correctly` with all the settings, there are a lot of `no active transaction found - cannot build new span`, but some `sending transaction` and `ending transaction` messages.

I have manually tampered the Secret Key, and there were authentication errors like `{"error":"invalid token"}`, so I can assume we are hitting a server.

The APM section won't show anything as also querying the index with this:

```auto
GET apm-*-error-*,apm-*-transaction-*/_search
{
  "size": 0, 
  "query": {
    "match_all": {}
  }
}

```

Any clues?

---

<div class="post-metadata">

**Author:** ![wa7son](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wa7son/32/21259_2.png) [@wa7son](https://discuss.elastic.co/u/wa7son)\
**Post date:** [June 28, 2019, 8:33am UTC](https://discuss.elastic.co/t/not-seeing-apm-data-on-elasticsearch-kibana/187911/2 "2019-06-28T08:33:36Z")

</div>

Hi Juan

Welcome to the forum 🙂

There can be several reasons why you don't see any data. First of all, unless you've configured the agent with [`metricsInterval: 0`](https://www.elastic.co/guide/en/apm/agent/nodejs/current/configuration.html#metrics-interval), you should be seeing data in the metrics index. Try to expand your search to include `apm-*-metric`.

For why you're not seeing any transactions, there can be several reasons. Before going into more details, I'd like to rule out the most common reason first: Transaction will only automatically be created if your application is an HTTP(S) server. If your application is exposing, for instance, a raw TCP API or some other protocol, or is pulling jobs off a message queue, you need to help the APM agent along by telling it when to start and end new transactions (see [`agent.startTransaction()`](https://www.elastic.co/guide/en/apm/agent/nodejs/current/agent-api.html#apm-start-transaction)).

Could this be the reason you don't see any transactions?

---

<div class="post-metadata">

**Author:** ![Juan\_Stiza](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/juan_stiza/32/45092_2.png) [@Juan\_Stiza](https://discuss.elastic.co/u/Juan_Stiza)\
**Post date:** [June 28, 2019, 12:10pm UTC](https://discuss.elastic.co/t/not-seeing-apm-data-on-elasticsearch-kibana/187911/3 "2019-06-28T12:10:32Z")

</div>

No luck with `apm-*-metric` and `metricsInterval` is set to 30.  
While our app may pull data from queues, it is primarily a simple Express server.  
I'll create a bare bones express server and test that to rule out software issues and let you know what I find.

---

<div class="post-metadata">

**Author:** ![Juan\_Stiza](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/juan_stiza/32/45092_2.png) [@Juan\_Stiza](https://discuss.elastic.co/u/Juan_Stiza)\
**Post date:** [June 28, 2019, 12:27pm UTC](https://discuss.elastic.co/t/not-seeing-apm-data-on-elasticsearch-kibana/187911/4 "2019-06-28T12:27:59Z")

</div>

Ok, here are my tests results, code is the next one:

```auto
const express = require('express');

const app = express();

app.get('/', (req, res, next) => {
  return res.send('Hello!');
});

app.listen(3000, () => console.log('Server up in port 3000'));

```

Installed libs:  
express@4.17.1  
elastic-apm-node@2.11.6

App output:

```auto
agent configured correctly { pid: 26687,
  ppid: 22454,
  arch: 'x64',
  platform: 'darwin',
  node: 'v8.12.0',
  agent: '2.11.6',
  ancestors:
   [ '/Users/juanstiza/dev/hunt/apm-test/node_modules/elastic-apm-node/index.js',
     '/Users/juanstiza/dev/hunt/apm-test/node_modules/elastic-apm-node/start.js',
     null,
     [length]: 3 ],
  startTrace:
   [ 'at Agent.start (/Users/juanstiza/dev/hunt/apm-test/node_modules/elastic-apm-node/lib/agent.js:149:11)',
     'at Object.<anonymous> (/Users/juanstiza/dev/hunt/apm-test/node_modules/elastic-apm-node/start.js:3:32)',
     'at Module._compile (module.js:653:30)',
     'at Object.Module._extensions..js (module.js:664:10)',
     'at Module.load (module.js:566:32)',
     'at tryModuleLoad (module.js:506:12)',
     'at Function.Module._load (module.js:498:3)',
     'at Module.require (module.js:597:17)',
     'at Function.Module._preloadModules (module.js:754:12)',
     'at preloadModules (bootstrap_node.js:488:38)',
     [length]: 10 ],
  main: undefined,
  dependencies: undefined,
  conf:
   { verifyServerCert: true,
     active: true,
     logLevel: 'trace',
     apiRequestSize: 786432,
     apiRequestTime: 10,
     stackTraceLimit: 50,
     captureExceptions: true,
     filterHttpHeaders: true,
     captureErrorLogStackTraces: 'messages',
     captureSpanStackTraces: true,
     captureBody: 'off',
     errorOnAbortedRequests: false,
     abortedErrorThreshold: 25,
     instrument: true,
     asyncHooks: true,
     sourceLinesErrorAppFrames: 5,
     sourceLinesErrorLibraryFrames: 5,
     sourceLinesSpanAppFrames: 0,
     sourceLinesSpanLibraryFrames: 0,
     errorMessageMaxLength: 2048,
     transactionMaxSpans: 500,
     transactionSampleRate: 1,
     serverTimeout: 30,
     disableInstrumentations: [[length]: 0 ],
     containerId: undefined,
     kubernetesNodeName: undefined,
     kubernetesNamespace: undefined,
     kubernetesPodName: undefined,
     kubernetesPodUID: undefined,
     captureHeaders: true,
     metricsInterval: 30,
     usePathAsTransactionName: false,
     addPatch: undefined,
     globalLabels: undefined,
     serviceName: 'hunt-mock-service',
     secretToken: '{{SecretToken}}',
     serverUrl: '{{ServerUrl}}',
     logger:
      { trace:
         { [Function: log]
           [length]: 0,
           [name]: 'log',
           [prototype]: log { [constructor]: [Circular] } },
        debug:
         { [Function: log]
           [length]: 0,
           [name]: 'log',
           [prototype]: log { [constructor]: [Circular] } },
        info:
         { [Function: log]
           [length]: 0,
           [name]: 'log',
           [prototype]: log { [constructor]: [Circular] } },
        warn:
         { [Function: log]
           [length]: 0,
           [name]: 'log',
           [prototype]: log { [constructor]: [Circular] } },
        error:
         { [Function: log]
           [length]: 0,
           [name]: 'log',
           [prototype]: log { [constructor]: [Circular] } },
        fatal:
         { [Function: log]
           [length]: 0,
           [name]: 'log',
           [prototype]: log { [constructor]: [Circular] } } },
     ignoreUrlStr: [[length]: 0 ],
     ignoreUrlRegExp: [[length]: 0 ],
     ignoreUserAgentStr: [[length]: 0 ],
     ignoreUserAgentRegExp: [[length]: 0 ],
     transport:
      { [Function: httpTransport]
        [length]: 2,
        [name]: 'httpTransport',
        [prototype]: httpTransport { [constructor]: [Circular] } },
     serverHost: '{{ServerUrl}}',
     serverPort: 443 } }
adding hook to Node.js module loader
shimming finalhandler@1.1.2 module
shimming http@8.12.0 module
shimming http.Server.prototype.emit function
shimming http.request function
shimming http.get function
shimming http.ServerResponse.prototype.writeHead function
shimming express@4.17.1 module
shimming express.Router.use function
shimming express.static function
copying property mime from express.static
shimming express.Router.Layer.handle function: query
shimming express.Router.Layer.handle function: expressInit
shimming express.Router.Layer.handle function: bound dispatch
Server up in port 3000
no active transaction found - cannot build new span
intercepted call to http.request { id: null }
intercepted request event call to http.Server.prototype.emit
start trace { trans: '58d24bca9fa7679b',
  parent: undefined,
  trace: 'c3f7fbba0d7a07a17cde4e56c1f2d4b1',
  name: null,
  type: null }
no original function off to wrap
no original function off to wrap
setting transaction result { trans: '58d24bca9fa7679b',
  parent: undefined,
  trace: 'c3f7fbba0d7a07a17cde4e56c1f2d4b1',
  result: 'HTTP 2xx' }
setting default transaction name: GET / { trans: '58d24bca9fa7679b',
  parent: undefined,
  trace: 'c3f7fbba0d7a07a17cde4e56c1f2d4b1' }
sending transaction { trans: '58d24bca9fa7679b',
  trace: 'c3f7fbba0d7a07a17cde4e56c1f2d4b1' }
ended transaction { trans: '58d24bca9fa7679b',
  parent: undefined,
  trace: 'c3f7fbba0d7a07a17cde4e56c1f2d4b1',
  type: 'request',
  result: 'HTTP 2xx',
  name: 'GET /' }
no active transaction found - cannot build new span
intercepted call to http.request { id: null }
no active transaction found - cannot build new span
intercepted call to http.request { id: null }
no active transaction found - cannot build new span
intercepted call to http.request { id: null }

```

Start script: `ELASTIC_APM_LOG_LEVEL=trace ELASTIC_APM_SERVICE_NAME=hunt-mock-service ELASTIC_APM_SECRET_TOKEN={{SecretToken}} ELASTIC_APM_SERVER_URL={{ServerUrl}} node -r elastic-apm-node/start index.js`

Still no dice.  
I tried setting the agent directly on the code:

```auto
require('elastic-apm-node').start({
  secretToken: '{{SecretToken}}',
  serverUrl: '{{ServerUrl}}',
  serviceName: 'hunt-mock-service'
});
const express = require('express');

const app = express();

app.get('/', (req, res, next) => {
  return res.send('Hello!');
});

app.listen(3000, () => console.log('Server up in port 3000'));

```

Same results.

---

<div class="post-metadata">

**Author:** ![Juan\_Stiza](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/juan_stiza/32/45092_2.png) [@Juan\_Stiza](https://discuss.elastic.co/u/Juan_Stiza)\
**Post date:** [June 28, 2019, 5:14pm UTC](https://discuss.elastic.co/t/not-seeing-apm-data-on-elasticsearch-kibana/187911/5 "2019-06-28T17:14:43Z")

</div>

I also tested restarting the APM server and reseting the Secret Token. Don't know what else to try.

---

<div class="post-metadata">

**Author:** ![Juan\_Stiza](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/juan_stiza/32/45092_2.png) [@Juan\_Stiza](https://discuss.elastic.co/u/Juan_Stiza)\
**Post date:** [June 28, 2019, 6:07pm UTC](https://discuss.elastic.co/t/not-seeing-apm-data-on-elasticsearch-kibana/187911/6 "2019-06-28T18:07:06Z")

</div>

Another update, we tested this same scenario on a new 7.2 install (we are testing prior migration) and it works:

![image](https://us1.discourse-cdn.com/elastic/original/3X/3/0/301cf00da4ee58f8c7a51c0975d044bb5091d6db.png)

Any clues why it wouldn't work with 6.6.1?

---

<div class="post-metadata">

**Author:** ![wa7son](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wa7son/32/21259_2.png) [@wa7son](https://discuss.elastic.co/u/wa7son)\
**Post date:** [July 1, 2019, 11:31am UTC](https://discuss.elastic.co/t/not-seeing-apm-data-on-elasticsearch-kibana/187911/7 "2019-07-01T11:31:44Z")

</div>

Interesting... Thanks for the updates and the test app. The agent should be just as compatible with 6.6.1 as it is with 7.2. But you might have found a compatibility bug we're not aware of. I'll have to dig a little deeper to see what's going on and get back to you.

Are you running both the APM Server, Elasticsearch, and Kibana all as v6.6.1?

---

<div class="post-metadata">

**Author:** ![Juan\_Stiza](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/juan_stiza/32/45092_2.png) [@Juan\_Stiza](https://discuss.elastic.co/u/Juan_Stiza)\
**Post date:** [July 1, 2019, 11:59am UTC](https://discuss.elastic.co/t/not-seeing-apm-data-on-elasticsearch-kibana/187911/8 "2019-07-01T11:59:16Z")

</div>

We have a v6.6.1 deployment:

![image](https://us1.discourse-cdn.com/elastic/original/3X/1/0/10f5f7d6f9de566c6a66006af03a76655f24d83b.png)

I am assuming all 3 services are the same version. Is there a place I can check that?

---

<div class="post-metadata">

**Author:** ![wa7son](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wa7son/32/21259_2.png) [@wa7son](https://discuss.elastic.co/u/wa7son)\
**Post date:** [July 1, 2019, 12:30pm UTC](https://discuss.elastic.co/t/not-seeing-apm-data-on-elasticsearch-kibana/187911/9 "2019-07-01T12:30:32Z")

</div>

Ah yes - they are. I forgot you were hosted on Elastic Cloud.

I'll look more into it...

---

<div class="post-metadata">

**Author:** ![Juan\_Stiza](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/juan_stiza/32/45092_2.png) [@Juan\_Stiza](https://discuss.elastic.co/u/Juan_Stiza)\
**Post date:** [July 1, 2019, 12:30pm UTC](https://discuss.elastic.co/t/not-seeing-apm-data-on-elasticsearch-kibana/187911/10 "2019-07-01T12:30:50Z")

</div>

Thanks!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 22, 2019, 8:30am UTC](https://discuss.elastic.co/t/not-seeing-apm-data-on-elasticsearch-kibana/187911/11 "2019-07-22T08:30:52Z")

</div>

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.