How to ingest ndjson into APM server?
We have an open telemetry data written (in ndjson) in the single file, which should be continuously loaded and sent to the APM.
We can’t use Filebeat as it has no APM output. We could use Logstash, as it has http output, but we are not sure about the output format. What about other possibilities?
Hi @Rnx ,
Other possibilities are covered in these docs, especially for ndjson → Elastic APM events intake API | Elastic Docs . You should be able to Ctrl+F for “ndjson”.
Try this article and let me know what you think. What output format would be ideal for your project?
Hello Sophia, thanks for the reply. In theory, the API intake works very well. In practice, you need a “sophisticated” tool to use it, as the command line tools such as “curl” are either terribly slow to use, or not reliable enough (dealing with network/server errors, etc.).
But, there is one solution to my problem - use the “OTEL Collector CONTRIB”, with otlp exporter. I am giving weight to word CONTRIB, as it is the only fork of OTEL suitable to handle the load and be reliable enough. GitHub - open-telemetry/opentelemetry-collector-contrib: Contrib repository for the OpenTelemetry Collector