Error does not display in APM Errors

[NoviceESCoder]

I have a asp.net core web app (mvc) application that writes logs using serilog. Whenever run I the application and perform operations and a certain operation that will cause an Error, that error will get recorded into Discover but when I go to APM Error section nothing is recorded. Is there something that is preventing that error not to be showing in the APM Errors tab?

You can see there is an error in the log stream:

Here you can see the log recorded in Discover:

But in the Errors tab that error is not displaying:

In the overview, you can see what it appears to be an error under error rate but when I click on it, nothing happens:

On the application console you can see the error too:

This controller is the one that handles the error:

namespace CustomerSimulatorApp.Controllers
{
    public class SecondController : Controller
    {

        private readonly ILogger<SecondController> _logger;

        public SecondController(ILogger<SecondController> logger)
        {
            _logger = logger;
        }

        public IActionResult SecIndex()
        { 
            return View();
        }

        [HttpPost]
        public IActionResult SecIndex(TextInput form)
        {
            try
            {
                if (ModelState.IsValid)
                    return RedirectToAction("FinalIndex", "Final");
                else
                    throw new Exception("Looks like you did not type something!");
             
            }
            catch(Exception ex)
            {
                _logger.LogError(ex, "Empty textfield!");
                return RedirectToAction("FinalIndex", "Final");
            }
        }     
    }
}

Is there something that is preventing the error not to be recorded into the Errors tab in APM?

[GregKalapos]

Hey on this thread as well @NoviceESCoder

The APM Agent will capture exceptions if you don't handle it - in this case you do handle it - so it won't be able to capture it.

On the other hand we have a module which will turn your logs that are on error level into APM errors, so it'll show up in the APM UI.

I remember from the other thread that you had this multiple agent initialization issue. This module which captures the error logs as APM errors is part of the setup that you enable on the IHostBuilder in Program.cs.

If you remove the UseAllElasticApm from Startup.cs and only leave it in Program.cs - do you still don't see these errors? In that case it should show up.

[NoviceESCoder]

Hey @GregKalapos

Oh I see, so if I do want to handle it, then it will never show up. Is there a workaround to get it to populate even if I do decide to handle that error?

Which module is that, that you are referring to?

Oh yes, I removed the UseAllElasticApm from Startup.cs and left it as so on Program.cs but it still does not show up

Let me show you those two files:

Startup.cs:

namespace CustomerSimulatorApp
{
    public class Startup
    {
        public Startup(IConfiguration configuration)
        {
            Configuration = configuration;
        }

        public IConfiguration Configuration { get; }

        // This method gets called by the runtime. Use this method to add services to the container.
        public void ConfigureServices(IServiceCollection services)
        {
            services.AddControllersWithViews();

            // create a new node instance
            var node = new Uri("http://localhost:9200");
            // settings instance for the node
            var settings = new ConnectionSettings(node);
            settings.DefaultFieldNameInferrer(p => p);

            services.AddSingleton<IElasticClient>(new ElasticClient(settings));
        }

        // This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
        public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
        {
          
            if (env.IsDevelopment())
            {
                app.UseDeveloperExceptionPage();
            }
            else
            {
                app.UseExceptionHandler("/Home/Error");
                // The default HSTS value is 30 days. You may want to change this for production scenarios, see https://aka.ms/aspnetcore-hsts.
                app.UseHsts();
            }

            app.UseHttpsRedirection();
            app.UseStaticFiles();

            app.UseRouting();

            app.UseAuthorization();

            app.UseEndpoints(endpoints =>
            {
                endpoints.MapControllerRoute(
                    name: "default",
                    pattern: "{controller=Home}/{action=Index}/{id?}");
            });
        }
    }
}

Program.cs:

namespace CustomerSimulatorApp
{
    public class Program
    {
        public static void Main(string[] args)
        {
            CreateHostBuilder(args).Build().Run();

            var logger = new LoggerConfiguration()
             .Enrich.WithElasticApmCorrelationInfo()
             .WriteTo.Console(outputTemplate: "[{ElasticApmTraceId} {ElasticApmTransactionId} {Message:lj} {NewLine}{Exception}")
             .CreateLogger();
        }


        public static IHostBuilder CreateHostBuilder(string[] args) =>
            Host.CreateDefaultBuilder(args)
                .UseSerilog((context, configuration) =>
                {
                    configuration.Enrich.FromLogContext()
                        .Enrich.WithElasticApmCorrelationInfo()
                        .Enrich.WithMachineName()
                        .WriteTo.Console()
                        .WriteTo.Elasticsearch(new ElasticsearchSinkOptions(new Uri(context.Configuration["ElasticConfiguration:Uri"]))
                        {
                            IndexFormat = $"{context.Configuration["ApplicationName"]}-logs-{context.HostingEnvironment.EnvironmentName?.ToLower().Replace(".", "-")}-{DateTime.UtcNow:yyyy-MM}",
                            AutoRegisterTemplate = true
                        })
                        .Enrich.WithProperty("Environment", context.HostingEnvironment.EnvironmentName)
                        .ReadFrom.Configuration(context.Configuration);
                })
                .ConfigureWebHostDefaults(webBuilder =>
                {
                    webBuilder.UseStartup<Startup>();
                })
                .UseAllElasticApm();
    }
}

On another note, inside of my Program.cs file, in the Main method, is it necessary to have the following in order for the .Enrich.WithElasticApmCorrelationInfo() to work that is inside of the CreateHostBuilder()?

   var logger = new LoggerConfiguration()
                 .Enrich.WithElasticApmCorrelationInfo()
                 .WriteTo.Console(outputTemplate: "[{ElasticApmTraceId} {ElasticApmTransactionId} {Message:lj} {NewLine}{Exception}")
                 .CreateLogger();

[NoviceESCoder]

@GregKalapos

Just wanted to keep you updated, I went ahead and tried to see what would happen if I did not handle the error by removing the return RedirectToAction("FinalIndex", "Final") from the catch and did a return new StatusCodeResult(500) which on the browser it gave me a This page does not work then in the errors tab, nothing showed up. Could there a be a reason why the error did not record in apm errors tab?

[GregKalapos]

The module which captures error logs as APM errors is this one: NuGet Gallery | Elastic.Apm.Extensions.Logging 1.25.1

This is part of the Elastic.Apm.NetCoreAll package and it's activated once you call .UseAllElasticApm() on IHostBuilder, which you do.

I created a super small sample here: GitHub - gregkalapos/LogAsErrorMiniSample

This is only APM - no log correlation since that's not relevant in this case. Maybe this helps you.

Here is how the log shows up as error:

Do you use the latest NuGet packages in your sample?

I went ahead and tried to see what would happen if I did not handle the error by removing the return RedirectToAction("FinalIndex", "Final") from the catch and did a return new StatusCodeResult(500) which on the browser it gave me a This page does not work then in the errors tab, nothing showed up. Could there a be a reason why the error did not record in apm errors tab?

This still won't capture you any error - it's important to note that an error in ASP.NET Core is typically an exception that you don't handle. In your example you manually set the status code to HTTP500 and return that - that isn't really an error from the APM's point of view - the ASP.NET Core pipeline in that case responds normally - so there is no exception that the agent could capture, it's just that the return code is 500. In that case you can capture an error manually with the Agent API, but the agent won't automatically capture an error for you, since technically there isn't any error there.

Having said that - your original example with the error log should work and in that case the agent would capture the log as an APM error, so it's worth looking at that. Can you maybe check my sample and see if that works for you and investigate from there?

[NoviceESCoder]

Hey @GregKalapos,

That's strange then for it not working if Elastic.Apm.Extensions.Logging is within Elastic.Apm.NetCoreAll. I downloaded your sample demo app and ran it and saw that it did record the error into the Errors tab. I do thank you for doing this for me. So when I went to my application where I have the

public IActionResult SecIndex(TextInput form)
        {
            try
            {
                if (ModelState.IsValid)
                    return RedirectToAction("FinalIndex", "Final");
                else
                    throw new Exception("Looks like you did not type something!");
             
            }
            catch(Exception ex)
            {
                _logger.LogError(ex, "Empty textfield!");
                return RedirectToAction("FinalIndex", "Final");
            }
        }     

I commented out the if & else and just left the return new Exception() and ran it and it still did not record it on to the Errors tab.

Which it looks like this now:

 public IActionResult SecIndex(TextInput form)
            {
                try
                {
                       throw new Exception("Looks like you did not type something!");    
                }
                catch(Exception ex)
                {
                    _logger.LogError(ex, "Empty textfield!");
                    return RedirectToAction("FinalIndex", "Final");
                }
            }     

So I did the if statement because I saw if you wanted to check if someone typed something into the textbox then you would need that condition statement, unless I am wrong?

If I leave the textbox empty and proceed with the button it will redirect me and on the exe file it shows up as Error. I noticed when I ran your sample it showed up as Fail. I am guessing that is why it is not recording?

Yes, I do have the latest NuGet packages in my application, please take a look.

On the errors tab I just see it blank...

Having said that - your original example with the error log should work and in that case the agent would capture the log as an APM error, so it's worth looking at that

I find it odd that it is not even after removing the if and else statement and just leaving the throw.