What is the best way to log via tcp?

Elastic Stack Logstash
1

Hey there,

I've been playing with the elastic stack for the past month and would like to implement it into my application.
The application is written in .NET and up until now I thought, that the communication via tcp would be the best way, considering the fact, that the udp communication was quiet easy to setup.

But I got a few problems and cant find anything to solve them. Which makes me suspicious, because my use case is not that.. unique. Is it?
The problem is, that either the log messages are registered as one big event, when I disconnect from the tcp server, or I cant use logs with multiple lines per event. (Because every line is registered as an event, by using "codec => line")

I use this Appender:
https://github.com/ugurozsahin/log4net-socket-appender, with these settings:
PatternLayout patternLayout = new PatternLayout();
patternLayout.ConversionPattern = "%date{dd/MM/yyyy hh:mm:ss,fff} | %thread | %level | %logger | %username | %P{log4net:HostName} | dev | %message | %exception | %newline";
patternLayout.ActivateOptions();

        TCPAppender appender = new TCPAppender();
        appender.RemoteAddress = ("192.168.1.180");
        appender.RemotePort = 55001;
        appender.AddressFamily = System.Net.Sockets.AddressFamily.InterNetwork;
        appender.SocketType = System.Net.Sockets.SocketType.Stream;
        appender.ProtocolType = System.Net.Sockets.ProtocolType.Tcp;
        appender.ConAttemptsCount = 5;
        appender.ConAttemptsWaitingTimeMilliSeconds = 3000;
        appender.UseThreadPoolQueue = false;
        appender.ActivateOptions();
        appender.Layout = patternLayout;

The patternlayout is completly flexible right now, as I am still in the development phase.

My LogStash Configuration:
input {
tcp{
port => 55001
codec => multiline {
# Grok pattern names are valid! :slight_smile:
pattern => "^%date{dd/MM/yyyy hh:mm:ss,fff}} "
negate => true
what => previous
}
type => "udp"
mode => "server"
}
udp
{
port => 55000
type => "udp"
}
}

Am I on the complete wrong path?

2

Okay, maybe I was a bit unclear in the beginning:
I am working on a .NET Application, which runs with multiple instances in my network. Those instances should log to logstash, which is the storing the data in es.

I began by using the log4net.UdpAppender, which worked great. I defined my own pattern layout and grok parsed it into field. Everything was perfect. BUT... nearly 10% of the data was lost under some stress. So I decided to switch to tcp.

After fining out, that log4net does not support a tcp appender, i used a community one:


As a last step, I switched the input from udp {port=>... } to tcp{port =>... mode => "server"}. Then the horror began.
My log messages would only be processed after the connection was canceld and then every line was summarized in one big ass event.
So I switched the codec to "lines" and put a newline after every message. This worked. BUT:
Whenever I send a multiple line stack trace, it gets messy. grokparsefailures and so on.

Up until now, I tried nearly everthing (multiline, (?m) at the beginning of the grok expression...) but nothing worked at.
What really creeps me out: I cant be the only one. This isnt really a unique situation, is it? Logging from .net over tcp to logstash?
Maybe I am completly wrong?

3

Don't use pattern layouts. Always serialize the raw log event objects in a standard way, preferably by using JSON. This eliminates issues with newline characters in fields, grok configurations, escaping, and so on.

Secondly I don't think sending log messages directly over the network is a good idea unless a) you know exactly what the failure mode is and b) you're willing to live with it. What I mean is, what happens if the network itself or your computer's network stack behaves erratically? Will you lose logs? Will the application hang? Something else?

I suggest you write the logs to disk and use Filebeat to ship them to Logstash.

What really creeps me out: I cant be the only one. This isnt really a unique situation, is it? Logging from .net over tcp to logstash?

I looked into this a couple of years ago (at a previous job) and couldn't find anything that I could use so I wrote a custom JSON formatter, had the logs written to disk, and configured NXLog to ship them to Logstash (this was before Filebeat).

1 Like
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.