What is the best approach to capture Linux(Ubuntu) server logs?

Elastic Stack Logstash
1

Can someone suggest me the best way to capture Ubuntu server logs and send it to Elasticsearch? I am trying to use Logstash but on top of that is there any other plugin? For windows event logs Winlogbeat and Logstash can be used. Like wise for Linux server logs, is there any other tool to use along with Logstash?
Thanks in advance.

2

You can use syslog-ng along with logstash.

3

Hi Makra, can you please say what disadvantages I will have if I use Logstash alone?

4

Depends on the requirement. If you have already syslog-ng server is running then you can use it to forward log to logstash. You can also convert log format ( e.g to JSON ) on the fly in syslog-ng itself. But this comes at the cost of one extra read/write operations. Rather you can use, filebeat/Winlogbeat to forward the same logs from the source. So basically, syslog-ng is used to collect logs from various sources in a centralized location and then forwards/distribute the logs to logstash for further refinement/processing which in turn may forward logs to ES->Kibana.

In general

Logs from n sources -> Syslog-ng Server -> Logstash -> ES -> Kibana

OR

logs from source 1 ->  Logstash -> ES -> Kibana

logs from source 2 ->  Logstash -> ES -> Kibana

logs from source 3 ->  Logstash -> ES -> Kibana

...............................................
...............................................

logs from source n ->  Logstash -> ES -> Kibana

Now you have to find it out, which one best suits your requirement.

5

Thanks for the detailed information Makra. Right now, we don't have Syslog-ng server running. So I will go with "Logs from n sources -> Syslog-ng Server -> Logstash -> ES -> Kibana". But will there be any overhead(CPU,Memory) on the server if I use only Logstash?

6

Any update on this please...

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.