Best way to visualize health and structure of data processing pipeline

Hello Florian,

Unfortunately, I do not have experiences with Spark nor with with Scala.

Form what I have read is that Scala is compiled to Java Bytecode so the Java agent might work - even if you might have to manually instrument the code if the used libraries are not yet supported(see here).

I do not know Spark but distributed environments are the place where APM shines as it shows the relations in a service map, shows calls to other services in the same Trace and so on. On each application installation you can configure the service name and the environment so you could have X installations of the same installation shown in the same entry in APM for Y different environments like DEV/QA/PROD. I cannot say how this shows in the Service Maps as we have no Platinum license which is required for the service maps.

Best regards
Wolfram