Unable to send logs to elasticsearch from .Net 8 using serilog when security is enabled

[Shahid_Hussain]

Hello,

I am unable to send logs to elasticsearch 8.12.2 when security is enabled on elasticsearch server. However, when I disable the security I can see the logs on elasticsearch server.

Below are the settings from elasticsearch.yml file.

Enable security features

xpack.security.enabled: true

xpack.security.enrollment.enabled: true

Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents

xpack.security.http.ssl:
enabled: true
keystore.path: certs/http.p12

This is my program.cs

using Serilog.Sinks.Elasticsearch;
using Serilog;
using System.Reflection;
using Serilog.Exceptions;
using Serilog.Sinks.RollingFile;
using Serilog.Formatting.Json;

var builder = WebApplication.CreateBuilder(args);

// Add services to the container.

builder.Services.AddControllers();
// Learn more about configuring Swagger/OpenAPI at https://aka.ms/aspnetcore/swashbuckle
builder.Services.AddEndpointsApiExplorer();
builder.Services.AddSwaggerGen();

configureLogging();
builder.Host.UseSerilog();

var app = builder.Build();

// Configure the HTTP request pipeline.
if (app.Environment.IsDevelopment())
{
    app.UseSwagger();
    app.UseSwaggerUI();
}

app.UseHttpsRedirection();

app.UseAuthorization();

app.MapControllers();

app.Run();


void configureLogging()
{
    var environment = Environment.GetEnvironmentVariable("ASPNETCORE_ENVIRONMENT");
    var configuration = new ConfigurationBuilder()
        .AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
        .AddJsonFile(
            $"appsettings.{environment}.json", optional: true
        ).Build();

    Log.Logger = new LoggerConfiguration()
        .Enrich.FromLogContext()
        .Enrich.WithExceptionDetails()
        .WriteTo.Elasticsearch(ConfigureElasticSink(configuration, environment))
        .Enrich.WithProperty("Environment", environment)
        .ReadFrom.Configuration(configuration) 
        .CreateLogger();
}



ElasticsearchSinkOptions ConfigureElasticSink(IConfigurationRoot configuration, string environment)
{
      return new ElasticsearchSinkOptions(new Uri(configuration["ElasticConfiguration:Uri"]))
        {
            ModifyConnectionSettings = x => x.BasicAuthentication("elastic", "password"),
            AutoRegisterTemplate = true,
            IndexFormat = $"{Assembly.GetExecutingAssembly().GetName().Name.ToLower().Replace(".", "-")}-{environment.ToLower()}-{DateTime.UtcNow:yyyy-MM}",
            NumberOfReplicas = 1,
            NumberOfShards = 2
        }; 
}

application.json

"Serilog": {
  "MinimumLevel": {
    "Default": "Information",
    "Override": {
      "Microsoft": "Information",
      "System": "Warning"
    }
  }
},
"ElasticConfiguration": {
  "Uri": "https://elastic:password@10.200.16.130:9200"
}

Any help would be greatly appreciated.

[carly.richmond]

Hi @Shahid_Hussain,

Are you seeing any errors in the logs at all? I would recommend not disabling the security settings.

[Shahid_Hussain]

Hi @carly.richmond ,

I don't find any error in the logs.

Shahid