OK, my root cause actually had nothing to do with Elastic.Apm. But I had to add some special FirstChanceException capturing code in my Program.cs in order to find out why Agent.Setup was failing, since no error was being returned other than something like "Agent initialization failed".
This led to the discovery of failures to access a version of System.Net.Http. First, problems finding the right .dll, then later problems finding the right version of the .dll.
Root cause was version mismatching of that library between a NuGet version(4.3.4) vs. the GAC version (4.2.0.0), plus some references across three projects that, when combined with that issue, just made it worse/more confusing.
Basically, installing or updating System.Net.Http via Nuget in a 4.7.2 or higher app can lead to these issues. With .Net version 4.7.2+, System.Net.Http is now a part of the .NET framework. You don't need the Nuget one.
Just thought I'd post this in case somebody else runs into this. (But, I know, this is such an old platform....)
Thanks!