Logstash process killed and as a result the ingestion stopped

[stillfreem]

Hi All,

I had a perfectly fine working configuration of Logstash.
I colleague of mine tried to troubleshoote and killed the Logstash process thought.
After Logstash restarted the ingestion stopped and in the log file I can only see the following error

[2022-09-09T08:54:28,341][INFO ][logstash.outputs.azureloganalytics] Resending 684 documents as log type github_auth to DataCollector API in 2 seconds.
[2022-09-09T08:54:30,455][INFO ][logstash.outputs.azureloganalytics] Resending 684 documents as log type github_auth to DataCollector API in 2 seconds.
[2022-09-09T08:54:32,710][INFO ][logstash.outputs.azureloganalytics] Resending 684 documents as log type github_auth to DataCollector API in 2 seconds.
[2022-09-09T08:54:34,854][INFO ][logstash.outputs.azureloganalytics] Resending 684 documents as log type github_auth to DataCollector API in 2 seconds.
[2022-09-09T08:54:37,066][INFO ][logstash.outputs.azureloganalytics] Resending 684 documents as log type github_auth to DataCollector API in 2 seconds.
[2022-09-09T08:54:39,197][ERROR][logstash.outputs.azureloganalytics] Could not resend 684 documents, message is dropped.
[2022-09-09T08:54:44,204][INFO ][logstash.outputs.azureloganalytics] Changing buffer size.[configuration='1700' , new_size='1600']
[2022-09-09T08:54:44,348][ERROR][logstash.outputs.azureloganalytics] Exception in posting data to Azure Loganalytics.
[Exception: '403 Forbidden]'

I'm pretty certain that it has nothing to do with authentication issue (as its says Forbidden 403), to me looks like a buffering issue.

My config is as follows

input
{
        #stdin{}

        syslog
        {
           host => "127.0.0.1"
           port => 5003
        }
}

filter
{

                if [PID] == "" { mutate { update => { "PID" =>  "N/A" }  } }
                else { mutate { gsub => [ "PID", "[\[\]]", ""] } }
                dissect { mapping => { "message" => "%{D} [%{Date}] %{LogLevel} %{} : %{data}" } }
                kv { source => "data" }
                mutate { add_tag => "kv" add_field => { "LogRelay" => "XXX" } }
                prune { blacklist_names => ["^data$", "now", "timestamp", "version", "logsource", "facility", "host", "enterprise", "tags", "elapsed", "priority", "worker_request_count"] }
                #prune { blacklist_values => ["^LogLevel$", "DEBUG"] }
}

#output
#{
#       stdout{}
#}

output
{
        microsoft-logstash-output-azure-loganalytics {
        workspace_id => "${W_ID}"
        workspace_key => "${W_KEY}"
        custom_log_table_name => "github_auth"
        }
}

The bad thing is that I send those logs to Log Analytics in Azure and the output plugin is not officially suported by Microsoft, so I was hoping that somebody can help me, please!!!