Auto instrumention - Metrics work, traces not received

Elastic Observability APM
,
21

Solution found!

The initial configuration consisted out of auto Profiler setup on the Application pools with the environment variables in the central /windows/microsoft.net/framework64/v4.0****/web.config

That setup works fine for stand-alone applications that do not use shared DLLs through a symbolic link.

For platforms that uses this setup you can solve it by only using the profiler setup like below:

$appcmd = "$($env:systemroot)\system32\inetsrv\AppCmd.exe"
$appPool = "OutSystemsApplications" 
$profilerHomeDir = "E:\ElasticStack\elastic_apm_profiler" 
$environment = @{
  COR_ENABLE_PROFILING = "1"
  COR_PROFILER_PATH = "$profilerHomeDir\elastic_apm_profiler.dll"
  COR_PROFILER = "{FA65FE15-F085-4681-9B20-95E04F6C03CC}"
  COMPlus_LoaderOptimization = "1" 
  ELASTIC_APM_PROFILER_HOME = "$profilerHomeDir\"
  ELASTIC_APM_PROFILER_INTEGRATIONS = "$profilerHomeDir\integrations.yml"
  ELASTIC_APM_PROFILER_LOG_DIR="$profilerHomeDir\logs"
  ELASTIC_APM_PROFILER_LOG="warn"
  ELASTIC_APM_PROFILER_LOG_IL="1"
  #Added agent environment settings below for the profiler to initialize the agent correctly!
  ELASTIC_APM_SERVER_URL="https://****.apm.westeurope.azure.elastic-cloud.com"
  ELASTIC_APM_SECRET_TOKEN="**********" 
  ELASTIC_APM_ENVIRONMENT="********"
  ELASTIC_APM_SERVICE_NAME="$appPool"
  ELASTIC_APM_CENTRAL_CONFIG="true"
  ELASTIC_APM_METRICS_INTERVAL="30000"
}

$environment.Keys | ForEach-Object {
  write-output "[name='$appPool'].environmentVariables.[name='$_',value='$($environment[$_])']"
  #delete entry
  & $appcmd set config -section:system.applicationHost/applicationPools /-"[name='$appPool'].environmentVariables.[name='$_']"
  #add entry
  & $appcmd set config -section:system.applicationHost/applicationPools /+"[name='$appPool'].environmentVariables.[name='$_',value='$($environment[$_])']" 
  
}
& $appcmd list config -section:system.applicationHost/applicationPools

It also simplifies our setup since I only need to change the Application pool name and rerun it..

@forloop Thanks for all the help!. Any chance our findings can be added to the installation documentation to help others?

Disadvantage: Can't use a service name per application anymore since it's configured centrally per app pool. But at least it's capturing, will see if we can find some way of identifying/grouping per app

22

Glad you got to the bottom of it, @Jbezemer :smiley:

It sounds like all applications within virtual directories, assigned to the same application pool, are sharing a single instance of the agent. Can you confirm if they use the same AppDomain?

The use of APM agent environment variables for the application pool means that the agent is configured the same way, irrespective of which application might instantiate the singleton instance first, which is what you would want I think.

The behaviour we're seeing here is related to the ASP.NET agent instrumentation, rather than the profiler auto instrumentation per se. That is, if we were to take the profiler out of the equation altogether, and configure each application through web.config to add the ElasticApmModule HttpModule, I'm sure we would see the same behaviour. More tests to assert behaviour under different IIS configurations will help, so that the documentation can provide better guidance for how to configure for different scenarios, or if certain scenarios are not officially supported. @GregKalapos, @alexander.wert - this may be something to add an issue for.

23

I believe they do all use the same AppDomain. I also believe that the current setup from your documentation works for most environments as long as they do not share the binaries by symbolic links, which is pretty common for low code platforms.

Applications on my environment in IIS that have their own binaries and no shared ones worked fine with the initial setup.

24

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.