How to configure Logstash conf file for multiples servers

Hi. I still didn't recognize from the documentation how I can configure Logstash to get logs from multiple servers. I have 2 servers with filebeat (send mysql and syslogs) on each server and a dedicated server for Logstash. In /etc/logstash/conf/d I have two files: mysql.conf and syslog.conf. In both files I wrote the port 5044 with host 0.0.0.0. How I need to configure these files in order to get logs from the second server?
If I leave these configurations and start the second filebeat I get the error :

Error: Address already in use
Exception: Java::JavaNet::BindException
Stack: sun.nio.ch.Net.bind0(Native Method)
sun.nio.ch.Net.bind(sun/nio/ch/Net.java:433)
sun.nio.ch.Net.bind(sun/nio/ch/Net.java:425)
sun.nio.ch.ServerSocketChannelImpl.bind(sun/nio/ch/ServerSocketChannelImpl.java:223)
io.netty.channel.socket.nio.NioServerSocketChannel.doBind(io/netty/channel/socket/nio/NioServerSocketChannel.java:128)
io.netty.channel.AbstractChannel$AbstractUnsafe.bind(io/netty/channel/AbstractChannel.java:558)
io.netty.channel.DefaultChannelPipeline$HeadContext.bind(io/netty/channel/DefaultChannelPipeline.java:1283)
io.netty.channel.AbstractChannelHandlerContext.invokeBind(io/netty/channel/AbstractChannelHandlerContext.java:501)
io.netty.channel.AbstractChannelHandlerContext.bind(io/netty/channel/AbstractChannelHandlerContext.java:486)
io.netty.channel.DefaultChannelPipeline.bind(io/netty/channel/DefaultChannelPipeline.java:989)
io.netty.channel.AbstractChannel.bind(io/netty/channel/AbstractChannel.java:254)
io.netty.bootstrap.AbstractBootstrap$2.run(io/netty/bootstrap/AbstractBootstrap.java:364)
io.netty.util.concurrent.AbstractEventExecutor.safeExecute(io/netty/util/concurrent/AbstractEventExecutor.java:163)
io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(io/netty/util/concurrent/SingleThreadEventExecutor.java:403)
io.netty.channel.nio.NioEventLoop.run(io/netty/channel/nio/NioEventLoop.java:463)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(io/netty/util/concurrent/SingleThreadEventExecutor.java:858)
io.netty.util.concurrent.FastThreadLocalRunnable.run(io/netty/util/concurrent/FastThreadLocalRunnable.java:30)
java.lang.Thread.run(java/lang/Thread.java:748)

I still didn't recognize from the documentation how I can configure Logstash to get logs from multiple servers.

You don't have to do anything. When Logstash listens for messages transmitted over the network it doesn't care how many clients are sending.

In /etc/logstash/conf/d I have two files: mysql.conf and syslog.conf. In both files I wrote the port 5044 with host 0.0.0.0.

No, you can't do that. Two listeners can't listen on the same port. Either use different ports or a single listener that you use for all inbound messages.

Unless you use Logstash's multi-pipeline feature, segregating your configuration into multiple files is only for your administrative convenience. To Logstash it's the same if you have mysql.conf and syslog.conf or if you merge those to files into a single file. Since you presumably want to have different filters for MySQL logs and for syslog messages you'll going to want to use conditionals to select which filters to apply to which messages.

Can you, please, give an example for the optimal configuration? The documentation isn't convenient enough to recognize how to get two kind of logs from multiple servers (in the future I need to get these logs from 100 servers approximately).

See https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html#conditionals and https://www.elastic.co/guide/en/logstash/current/config-examples.html for several examples of how to use conditionals to selectively apply filters.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.