No Traces or Transactions showing up in Kibana

Elastic Observability APM
1

Posting raw JSON into the APM. Not using an agent as I am trying to work with API.

Using latest 7.4 version of ES, Kiban and APM.

Browser version: Chrome v77

Original install method: yum, fresh install

Posted the following JSON and got a 200 back.

{"metadata":{"service":{"name":"My Service","agent":{"name":"My Test Agent","version":"1.0"},"language":{"name":"Never Mind","version":"Seattle"}},"process":{"pid":13756,"title":"APMTest","argv":},"system":{"architecture":"x64","hostname":"PCW7PRODEV","platform":"Windows"},"user":{"id":"S-1-5-21-4160314693-201202386-2634938281-1000","username":"PCW7PRODEV\ntavendale"}}}
{"span":{"id":"550A73C55F1A9CE0","trace_id":"550A73C55F1A9CE0037114652EE5BBFB","parent_id":"550A73C55F1A9CE0","transaction_id":"550A73C55F1A9CE0","parent":1,"name":"GET /","type":"http request","start":0.0186281284739957,"duration":392.280143015954,"stacktrace":,"context":{"http":{"url":"http://www.amazon.com","status_code":301,"method":"GET"}}}}
{"span":{"id":"73870394282827EB","trace_id":"550A73C55F1A9CE0037114652EE5BBFB","parent_id":"550A73C55F1A9CE0","transaction_id":"550A73C55F1A9CE0","parent":1,"name":"GET /","type":"http request","start":392.304479764444,"duration":95.692395517231,"stacktrace":,"context":{"http":{"url":"http://www.allthingssyslog.com","status_code":200,"method":"GET"}}}}
{"span":{"id":"29571EA55FB1313D","trace_id":"550A73C55F1A9CE0037114652EE5BBFB","parent_id":"550A73C55F1A9CE0","transaction_id":"550A73C55F1A9CE0","parent":1,"name":"GET /","type":"http request","start":488.010095243818,"duration":1.80332301775681,"stacktrace":,"context":{"http":{"url":"http://127.0.0.1","status_code":404,"method":"GET"}}}}
{"error":{"id":"73870394282827EB","timestamp":1570912725000000,"trace_id":"550A73C55F1A9CE0037114652EE5BBFB","transaction_id":"550A73C55F1A9CE0","parent_id":"550A73C55F1A9CE0","culprit":"TTransactionWithSpansTest.Load","exception":{"code":"0","message":"HTTP/1.1 301 Moved Permanently","attributes":null,"stacktrace":null,"type":"EIdHTTPProtocolException","handled":true}}}
{"error":{"id":"29571EA55FB1313D","timestamp":1570912725000000,"trace_id":"550A73C55F1A9CE0037114652EE5BBFB","transaction_id":"550A73C55F1A9CE0","parent_id":"550A73C55F1A9CE0","culprit":"TTransactionWithSpansTest.Load","exception":{"code":"0","message":"HTTP/1.1 404 Not Found","attributes":null,"stacktrace":null,"type":"EIdHTTPProtocolException","handled":true}}}
{"transaction":{"trace_id":"550A73C55F1A9CE0037114652EE5BBFB","id":"550A73C55F1A9CE0","type":"request","duration":489.814019168945,"timestamp":1570912715000000,"result":"TRUE","spans":null,"sampled":null,"span_count":{"started":3,"dropped":0},"context":{"system":{"architecture":"x64","hostname":"PCW7PRODEV","platform":"Windows"}}}}

2

Hi Nigel, welcome to the forum!

Did you have a look at the Intake API documentation? Since it doesn't render well sub-documents, you can see everything in the GitHub repo.
You must fully adhere to the API in order to make this work. For example, I see you only sent timestamps for the errors and transaction, but not for the spans.

If I may ask- why are you sending data this way? Is it for creating sample data or are you developing some kind of an agent?

3

If I may ask- why are you sending data this way?

I'm developing an agent. I don't use any of the languages you have agents for now, so I have to do it myself.

For example, I see you only sent timestamps for the errors and transaction, but not for the spans.

The spans have a start and duration (Spans | APM User Guide [8.11] | Elastic), unless you are referring to { "$ref": "../timestamp_epoch.json" } that appears in the Intake API docs.
Also the span in the example request body (https://www.elastic.co/guide/en/apm/server/master/example-intake-events.html) does not have a timestamp.

4

Out of curiosity, for which language are you developing an agent?

The timestamp is optional and will be set to the time when the APM Server receives the events, so it's recommended to explicitly set the timestamp.

Maybe there's anything in the APM Server logs which indicates where the events get lost?

5

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.