APM-Server not connecting to my application

[Piyush_Mittal]

Hey, I just started using APM to monitor my application. I done successful setup of APM-server along with my nodejs apm-agent on my local machine. But, When I go to setup the same for production, my APM-server is not connecting to my application.

I created a simple nodejs application and create it's docker image and run the docker container in my AWS EC2 instance and then install Elasticsearch, Kibana and APM-Server on my instance and everything is working fine and I am also successfully entered into my dashboard but When I try to launch APM, It's not launching it. And when I check, I found my apm-server is running successfully on my ec2 and my application also but they are not connecting with each other. I don't know why this happen on production. Beacuse same thing is working fine on my machine. So, Please tell me How I can solve this issue??

If you guy's need any info regarding configuration, please tell. I will share here.

I am attaching my nodejs code here, pls tell if u need anything else to solve the issue.

  serviceName: "nodeAPMSerivce",
  serverUrl: "http://localhost:8200",
  metricsInterval: "0s",
});

const express = require("express");
const connectDB = require("./config/db");
const dotenv = require("dotenv");
const User = require("./models/userModel");

dotenv.config();

const app = express();

connectDB();

app.use(express.json());

app.get("/", (req, res) => {
  res.send("Hey, I'm here!!");
});

app.get("/users", async (req, res) => {
  try {
    const users = await User.find({});

    res.status(200).json({
      users,
    });
  } catch (err) {
    console.log(err);
  }
});

app.post("/user", async (req, res) => {
  try {
    const savedUser = await User.create({ ...req.body });

    res.status(201).json({
      savedUser,
    });
  } catch (err) {
    console.log(err);
  }
});

app.listen(8080, () => {
  console.log("Hello I'm listening on PORT 8080");
});

Thanks

[simitt (Silvia Mitter)]

Your configuration shows serverUrl: "http://localhost:8200", which only works as long as your APM instrumented app and the APM Server run alongside on the same host. Please ensure that the configured APM Server URL is in fact reachable from within your app. The agent logs should provide more information about whether or not the server can be reached.

I suggest to also consider setting up an auth.secret_token for the APM Server and configure it within the Node.js app.

[Piyush_Mittal]

@simitt I checked in my apm-server.yml file host is localhost:8200 and my apm-server is also running.

As you can see in screenshot that APM server is running:

The issues which I am facing is:

• My app which is running on same instance unable to connect to my apm-server:

• In my apm-server.yml If I uncomment my elasticsearch host then my apm-server running failed.

Can you Please tell me what is the issue here and How I can solve this. Please tell.

Thanks.

[Piyush_Mittal]

@axw and @trentm Please tell a solution of this issue, please help me to to came out of this issue.

[trentm (Trent Mick)]

@Piyush_Mittal

My app which is running on same instance unable to connect to my apm-server

@Piyush_Mittal Your app and apm-server may be running on the same EC2 instance, but if they are running in separate docker containers, then they have separate networks for "localhost". Often (if using docker "bridge" networking) docker containers will be run with -p 8200 or similar to expose ports from the container to the docker host (Docker run reference | Docker Docs). If your app is also running in a Docker container on that same EC2 host, then you'll want to setup a docker network (docker network | Docker Docs) all your containers can share to talk with each other.

See Docker run reference | Docker Docs.

As @simitt mentioned above, please confirm from where your app is running (e.g. if it is in a separate container), that it can reach apm-server.

[Piyush_Mittal]

@trentm I have one EC2 instance on which my docker container and apm-server, elasticsearch and kibana is running.

The structure is like:-

1. One docker container on which my nodejs application is running.

2. My apm-server, elastic search and kibana is running on my ec2 machine not in any docker container.

SO, I need to setup any network bridge or not ??

Thanks.

[trentm (Trent Mick)]

@Piyush_Mittal You could try using "host" networking for your application's container by starting it with docker run --network=host .... See Use host networking | Docker Documentation

[Piyush_Mittal]

@trentm Thanks for your help, I will definitely try your solution and If I got any error I will get back here.

[Piyush_Mittal]

@trentm I exposed 8200 post in my docker file and then run this command sudo docker run -d -p 8080:8080 -p 8200:8200 --name elkContainer piyush2002/elkapplication:latest and got his error:-

And I also used this command sudo docker run -d --network=host --name elkContainer piyush2002/elkapplication:latest but my apm agent not started as I checked logs of my application.

Please tell what else I can do.

[Piyush_Mittal]

@trentm I am also sharing my DockerFile:-

FROM node:alpine

WORKDIR /usr/app

COPY package.json .

RUN npm install

COPY . .

EXPOSE 8080

EXPOSE 8200

CMD ["npm", "run", "start"]

Please tell if any changes needed.

[Piyush_Mittal]

Hey, @trentm Pleaseeee try to reply as soon as possible.

[nugusbayevkk (nugusbayevk)]

Hi @Piyush_Mittal , why are you trying expose 8200 port? If you look at error message, this port is already in use.
How did you run apm-server, elasticsearch?

[trentm (Trent Mick)]

@Piyush_Mittal As @nugusbayevkk has mentioned, you don't want to "EXPOSE 8200" from your application's Dockerfile, because port 8200 is the port used by apm-server.

[Piyush_Mittal]

@trentm and @nugusbayevkk Initially, I just trying it with not exposing the port then it's not working and I also use --network=host as suggested by @trentm but if I use this then my apm-agent is not starting in my application. And I also clone my application on my ec2 machine and run it and it still no connecting to apm-server. This thing not happened with me when I run my application locally and also my apm-server on my PC. I don't know like What's wrong going in this. Please help me out like if you want to me to share some more details about this stuff , I am more than happy to share with you but please help me to figure this out.