Cannot view logs in Observability log app

I have a asp.net core web app (mvc) project that will write the logs using Serilog to Elasticsearch. I created the index pattern in Kibana and I can view the logs that was written to elasticsearch under Discover. But when I go to Observability -> Logs it shows there are no log messages to display. I go to settings, then under Log indices I include the index pattern for matching indices that contain log data which for me is customer-simulation-es-app-logs* and I hit apply. But when I go back to view the logs, nothing will display even after I run the program and write logs to elasticsearch. I can only view the logs in discover but nothing will display in Logs.

Is there a way to fix this issue so I can view the logs that I have written to elasticsearch under logs in Observability?

appsettings.json

{
  "ApplicationName": "customer-simulation-es-app",
  "Serilog": {
    "MinimumLevel": {
      "Default": "Information",
      "Override": {
        "Microsoft": "Information",
        "System": "Warning"
      }
    }
  },
  "ElasticConfiguration": {
    "Uri": "http://localhost:9200"
  },
  "AllowedHosts": "*"
}

Program.cs

public static IHostBuilder CreateHostBuilder(string[] args) =>
            Host.CreateDefaultBuilder(args)
                .UseSerilog((context, configuration) =>
                {
                    configuration.Enrich.FromLogContext()
                        .Enrich.WithMachineName()
                        .WriteTo.Console()
                        .WriteTo.Elasticsearch(new ElasticsearchSinkOptions(new Uri(context.Configuration["ElasticConfiguration:Uri"]))
                        {
                            IndexFormat = $"{context.Configuration["ApplicationName"]}-logs-{context.HostingEnvironment.EnvironmentName?.ToLower().Replace(".", "-")}-{DateTime.UtcNow:yyyy-MM}",
                            AutoRegisterTemplate = true
                        })
                        .Enrich.WithProperty("Environment", context.HostingEnvironment.EnvironmentName)
                        .ReadFrom.Configuration(context.Configuration);
                })
                .ConfigureWebHostDefaults(webBuilder =>
                {
                    webBuilder.UseStartup<Startup>();
                });

This is what keeps displaying but there is logs... even if I click on stream live and perform the operations in the app, nothing will get displayed here but in Discover the logs do display. Not sure why I cannot view my logs under Observability -> Logs.

enter image description here

This is the logs in Discover:

enter image description here

Hi @NoviceESCoder,

Sorry to see you're having problems. There isn't anything obvious wrong here, and we would have expected some errors to have surfaced before this point if certain things were wrong. So we'll try to figure out what might be happening here.

Please could you run the following command in the dev tools within Kibana and paste the output here:

GET /customer-simulation-es-app-logs*/_mapping

This will allow us to see the exact mappings for those indices.

Could you also check your Kibana logs to see if anything may have surfaced there.

Lastly, could you please check the network panel of your browser's developer tools to see if there are any failed requests.

Sure thing, I greatly appreciate your help.

This is what I get when I run the cmd GET /customer-simulation-es-app-logs*/_mapping. I attempted to copy the entire result but it is too large and does not let me post. So here are the pictures.

















I get these two errors when I inspect the page:

Hmm, mappings look good, thank you.

The errors in the console are okay (well, unrelated).

Did you see anything in your Kibana server logs at all?

Also, could you check if there's any errors in your network panel specifically (not console) in the developer tools. Please could you check this on the Log's Stream page.

Lastly, generating a HAR recording of the network panel and sharing that could be useful. This would again be taken on the Log's Stream page.

I think there was a bug going on, because I am now able to see the logs! Very strange it was not working for 3 days until just now! Not sure what caused it not to work, but its working! Thanks for your help!!

Glad you're sorted :slight_smile: