# Elastic APM Agent Getting HTTP 404 **URL:** \ **Category:** Kibana\ **Created:** [February 5, 2019, 3:00am UTC](https://discuss.elastic.co/t/elastic-apm-agent-getting-http-404/167052 "2019-02-05T03:00:35Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![harry2190](https://avatars.discourse-cdn.com/v4/letter/h/e68b1a/32.png) [@harry2190](https://discuss.elastic.co/u/harry2190)\ **Post date:** [February 5, 2019, 3:00am UTC](https://discuss.elastic.co/t/elastic-apm-agent-getting-http-404/167052/1 "2019-02-05T03:00:35Z")
Hi, We have setup a java agent in our server running the java base CA7 API Gateway. Reference Log: 2019-01-31T18:50:41.279+0800 INFO 9 STDOUT : 2019-01-31 18:50:41.279 [apm-reporter] ERROR co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Error sending data to APM server: http://:8200/intake/v2/events, response code is 404 2019-01-31T18:50:41.280+0800 INFO 9 STDOUT : 2019-01-31 18:50:41.280 [apm-reporter] WARN co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - \ Reference Server Details: - Agent using 1.3.0 - APM Server using v6.5.4 - CA7 Server running Java build 1.8.0\_73-b02 Additional Details in Kibana: - Dashboard was able to get that an agent is running but, no additional details of transactions made is showing in the server Requesting kindly for your guidance if you have encountered this issues as well. ![image](https://us1.discourse-cdn.com/elastic/original/3X/7/d/7d4a16b92e62eb7ed2b715915e21985142ee85dc.jpeg) ---
**Author:** ![jalvz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jalvz/32/45613_2.png) [@jalvz](https://discuss.elastic.co/u/jalvz)\ **Post date:** [February 5, 2019, 8:46pm UTC](https://discuss.elastic.co/t/elastic-apm-agent-getting-http-404/167052/2 "2019-02-05T20:46:48Z")
Hi! That URL in the error looks very suspicious, did you edited it to hide the host or you get that error exactly? (`http://:8200/intake/v2/events`) For reference, here it says how you can specify the APM Server URL: [https://www.elastic.co/guide/en/apm/agent/java/current/configuration.html](https://www.elastic.co/guide/en/apm/agent/java/current/configuration.html) Does that help? ---
**Author:** ![harry2190](https://avatars.discourse-cdn.com/v4/letter/h/e68b1a/32.png) [@harry2190](https://discuss.elastic.co/u/harry2190)\ **Post date:** [February 6, 2019, 1:45am UTC](https://discuss.elastic.co/t/elastic-apm-agent-getting-http-404/167052/3 "2019-02-06T01:45:10Z")
Hi javlz, Thank you for the reply. We deliberately edited the host in this post. The is actually a server ip address that we have. Do note that we also did try health checks, ping to the server and it all returns fine. ---
**Author:** ![felixbarny](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/felixbarny/32/27341_2.png) [@felixbarny](https://discuss.elastic.co/u/felixbarny)\ **Post date:** [February 6, 2019, 6:46am UTC](https://discuss.elastic.co/t/elastic-apm-agent-getting-http-404/167052/4 "2019-02-06T06:46:44Z")
> [@harry2190](#): > > 2019-01-31T18:50:41.279+0800 INFO 9 STDOUT : 2019-01-31 18:50:41.279 [apm-reporter] ERROR co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Error sending data to APM server: http://:8200/intake/v2/events, response code is 404 > > 2019-01-31T18:50:41.280+0800 INFO 9 STDOUT : 2019-01-31 18:50:41.280 [apm-reporter] WARN co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - \ The message `` hints that the `server_urls` you have configured is a URL of a WebLogic application server. Did you set it to the application you want to monitor? It is supposed to be the APM Server URL. I hope this helps 🙂 Felix ---
**Author:** ![harry2190](https://avatars.discourse-cdn.com/v4/letter/h/e68b1a/32.png) [@harry2190](https://discuss.elastic.co/u/harry2190)\ **Post date:** [February 6, 2019, 7:33am UTC](https://discuss.elastic.co/t/elastic-apm-agent-getting-http-404/167052/5 "2019-02-06T07:33:03Z")
Hi felixbarny, Thank you for your response. The ip address we have put in the java arguments is the actual APM Server IP. We even check the logs and it shows on health check status that it is ok. 2019-02-06T12:12:35.041+0800 INFO 6 STDOUT : 2019-02-06 12:12:35.041 [apm-server-healthcheck] INFO co.elastic.apm.agent.report.ApmServerHealthChecker - Elastic APM server is available: {"ok":{"build\_date":"2018-12-17T20:15:18Z","build\_sha":"fd4f1ac006d8896e4957b6c01fb1e9b9ff04afd7","version":"6.5.4"}} When it do try to http://APM\_SERVER\_IP:8200/intake/v2/events it is getting the HTTP 404 with the HTML response. ---
**Author:** ![felixbarny](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/felixbarny/32/27341_2.png) [@felixbarny](https://discuss.elastic.co/u/felixbarny)\ **Post date:** [February 6, 2019, 10:11am UTC](https://discuss.elastic.co/t/elastic-apm-agent-getting-http-404/167052/6 "2019-02-06T10:11:28Z")
What happens if you do `curl -XPOST http://APM_SERVER_IP:8200/intake/v2/events`? Could it be that your APM Server is behind a proxy and it does not forward the `intake/v2/events` path? ---
**Author:** ![harry2190](https://avatars.discourse-cdn.com/v4/letter/h/e68b1a/32.png) [@harry2190](https://discuss.elastic.co/u/harry2190)\ **Post date:** [February 6, 2019, 11:03am UTC](https://discuss.elastic.co/t/elastic-apm-agent-getting-http-404/167052/7 "2019-02-06T11:03:39Z")
Hi Felix, Doing a curl actually return a non HTTP 404 response. curl -v -X POST http://API\_SERVER\_IP:8002 About to connect() to API\_SERVER\_IPport 8200 (#0) - Trying API\_SERVER\_IP... connected - Connected to API\_SERVER\_IP (API\_SERVER\_IP) port 8200 (#0) > POST /intake/v2/events HTTP/1.1 > User-Agent: curl/7.19.7 (x86\_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.27.1 zlib/1.2.3 libidn/1.18 libssh2/1.4.2 > Host: APM\_SERVER\_IP:8200 > Accept: _/_ \< HTTP/1.1 400 Bad Request \< Connection: Close \< Content-Type: application/json \< Date: Wed, 06 Feb 2019 11:00:45 GMT \< Content-Length: 64 \< - Closing connection #0 {"accepted":0,"errors":[{"message":"invalid content type: ''"}]} To answer about the APM Server could be behind proxy, we honestly believe that is not the case since other servers using Java was able to send data in intake/v2/events. ---
**Author:** ![felixbarny](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/felixbarny/32/27341_2.png) [@felixbarny](https://discuss.elastic.co/u/felixbarny)\ **Post date:** [February 6, 2019, 11:58am UTC](https://discuss.elastic.co/t/elastic-apm-agent-getting-http-404/167052/8 "2019-02-06T11:58:59Z")
That curl response looks like it should. Could it be that you added proxy settings to the JVM (`-Dhttp.proxyHost` system property)? See [https://docs.oracle.com/javase/6/docs/technotes/guides/net/proxies.html](https://docs.oracle.com/javase/6/docs/technotes/guides/net/proxies.html) If so, adding `-Dhttp.noProxyHosts="APM_SERVER_IP"` would probably help. ---
**Author:** ![harry2190](https://avatars.discourse-cdn.com/v4/letter/h/e68b1a/32.png) [@harry2190](https://discuss.elastic.co/u/harry2190)\ **Post date:** [February 7, 2019, 1:19am UTC](https://discuss.elastic.co/t/elastic-apm-agent-getting-http-404/167052/9 "2019-02-07T01:19:29Z")
Hi Felix, Thank you for the input. Will check on this. 😄 ---
**Author:** ![harry2190](https://avatars.discourse-cdn.com/v4/letter/h/e68b1a/32.png) [@harry2190](https://discuss.elastic.co/u/harry2190)\ **Post date:** [February 7, 2019, 3:29am UTC](https://discuss.elastic.co/t/elastic-apm-agent-getting-http-404/167052/10 "2019-02-07T03:29:09Z")
Hi Felix, We tried to check if any of the running JVM in the server has the -Dhttp.proxyHost argument. Somehow we don't see it. /opt/SecureSpan/Appliance/libexec/gateway\_control pc run -J-Dcom.l7tech.server.home="/opt/SecureSpan/Gateway/node/default" -J-Dcom.l7tech.server.processControllerPresent=true -J-Dcom.l7tech.server.processControllerCert=/opt/SecureSpan/Controller/var/run/pc.cer -J-Dcom.l7tech.server.processControllerIpAddress=127.0.0.1 -J-Dcom.l7tech.server.processControllerPort=8765 -J-Djava.util.logging.config.class=com.l7tech.server.log.JdkLogConfig -J-Dcom.l7tech.server.log.console=true /bin/bash /opt/SecureSpan/Appliance/libexec/gateway\_control pc run -J-Dcom.l7tech.server.home="/opt/SecureSpan/Gateway/node/default" -J-Dcom.l7tech.server.processControllerPresent=true -J-Dcom.l7tech.server.processControllerCert=/opt/SecureSpan/Controller/var/run/pc.cer -J-Dcom.l7tech.server.processControllerIpAddress=127.0.0.1 -J-Dcom.l7tech.server.processControllerPort=8765 -J-Djava.util.logging.config.class=com.l7tech.server.log.JdkLogConfig -J-Dcom.l7tech.server.log.console=true /opt/SecureSpan/JDK/bin/java -Dcom.l7tech.server.home=/opt/SecureSpan/Gateway/node/default -Djava.ext.dirs=/opt/SecureSpan/JDK/jre/lib/ext:/opt/SecureSpan/Gateway/runtime/lib/ext -Dcom.l7tech.server.siteminder.enabled=true -server -Djava.net.preferIPv4Stack=true -Djava.security.policy=/opt/SecureSpan/Gateway/runtime/etc/ssg.policy -Djava.security.egd=file:/dev/./urandom -Dfile.encoding=UTF-8 -Dsun.net.inetaddr.ttl=30 -Djava.awt.headless=true -XX:CompileThreshold=1500 -Dcom.l7tech.server.defaultClusterHostname=[CA7-AP-DEV.hostname.net](http://CA7-AP-DEV.hostname.net) -Djava.util.Arrays.useLegacyMergeSort=true -Xmx20232394k -Xss256k -XX:+UseParallelOldGC -XX:+UseCompressedOops -Dcom.l7tech.server.transport.reservedPorts=7001,7100 -javaagent:/home/gateway/elastic-apm-agent-1.3.0.jar -Delastic.apm.service\_name=dev-ca7 -Delastic.apm.application\_packages=. -Delastic.apm.server\_url=http://APM\_SERVER\_IP:8200 -XX:MaxPermSize=256M -Dcom.l7tech.server.home="/opt/SecureSpan/Gateway/node/default" -Dcom.l7tech.server.processControllerPresent=true -Dcom.l7tech.server.processControllerCert=/opt/SecureSpan/Controller/var/run/pc.cer -Dcom.l7tech.server.processControllerIpAddress=127.0.0.1 -Dcom.l7tech.server.processControllerPort=8765 -Djava.util.logging.config.class=com.l7tech.server.log.JdkLogConfig -Dcom.l7tech.server.log.console=true -jar /opt/SecureSpan/Gateway/runtime/Gateway.jar pc run /bin/bash /opt/SecureSpan/Appliance/libexec/gateway\_control pc run -J-Dcom.l7tech.server.home="/opt/SecureSpan/Gateway/node/default" -J-Dcom.l7tech.server.processControllerPresent=true -J-Dcom.l7tech.server.processControllerCert=/opt/SecureSpan/Controller/var/run/pc.cer -J-Dcom.l7tech.server.processControllerIpAddress=127.0.0.1 -J-Dcom.l7tech.server.processControllerPort=8765 -J-Djava.util.logging.config.class=com.l7tech.server.log.JdkLogConfig -J-Dcom.l7tech.server.log.console=true ---
**Author:** ![felixbarny](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/felixbarny/32/27341_2.png) [@felixbarny](https://discuss.elastic.co/u/felixbarny)\ **Post date:** [February 7, 2019, 9:10am UTC](https://discuss.elastic.co/t/elastic-apm-agent-getting-http-404/167052/11 "2019-02-07T09:10:31Z")
Have you tried setting `-Dhttp.noProxyHosts="APM_SERVER_IP"`? To get some more insight, you could enable logging for HttpUrlConnection: [https://stackoverflow.com/a/12339718/1125055](https://stackoverflow.com/a/12339718/1125055) That should also log any proxies used for the connection. ---
**Author:** ![harry2190](https://avatars.discourse-cdn.com/v4/letter/h/e68b1a/32.png) [@harry2190](https://discuss.elastic.co/u/harry2190)\ **Post date:** [February 8, 2019, 5:45am UTC](https://discuss.elastic.co/t/elastic-apm-agent-getting-http-404/167052/12 "2019-02-08T05:45:31Z")
Hi Felix, Yes, we had tried putting the argument but, same issue still persist. As additional step for troubleshooting we had installed the APM Server in same machine as the CA7 API Gateway. We also get same error (HTTP 404) when we point to localhost server of APM. ---
**Author:** ![felixbarny](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/felixbarny/32/27341_2.png) [@felixbarny](https://discuss.elastic.co/u/felixbarny)\ **Post date:** [February 8, 2019, 10:01am UTC](https://discuss.elastic.co/t/elastic-apm-agent-getting-http-404/167052/13 "2019-02-08T10:01:12Z")
It seems like the requests to the APM server are proxied through the CA7 API Gateway. CA7 runs on WebLogic which prints `` on 404 pages. You need to either make sure the `/intake/v2/events` path is proxied correctly as well or to remove the proxy for the APM Server. ---
**Author:** ![harry2190](https://avatars.discourse-cdn.com/v4/letter/h/e68b1a/32.png) [@harry2190](https://discuss.elastic.co/u/harry2190)\ **Post date:** [February 11, 2019, 2:43am UTC](https://discuss.elastic.co/t/elastic-apm-agent-getting-http-404/167052/14 "2019-02-11T02:43:00Z")
Hi Felix, Let us try to check out your suggestion to our infra team. Hopefully this help us resolve the http 404. 🙂 ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [March 11, 2019, 2:47am UTC](https://discuss.elastic.co/t/elastic-apm-agent-getting-http-404/167052/15 "2019-03-11T02:47:52Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.