Logstash syslog input from two sources

Announcements Community Ecosystem
1

Hi All

I am trying to ingest syslog from two separate sources (one firewall and one Linux based appliance) and forward to log analytics workspace in Azure. I was able to achieve this using two separate conf files each with unique ports (one running on port 514 and other on 515). The problem is when I run two conf together it doesn't work. I am new to logstash and any help with be greatly appreciated.

My conf files look like this
Conf file 1

input {
     syslog {
         port => 514
    }
}

output {
    microsoft-sentinel-logstash-output {
      managed_identity => true
      tenant_id => "XXX"
	  data_collection_endpoint => "https://XXX1.uksouth-1.ingest.monitor.azure.com"
      dcr_immutable_id => "dcr-XXX1"
      dcr_stream_name => "Custom-XXX1"
    }
}

Conf file 2

input {
     syslog {
         port => 515
    }
}

output {
    microsoft-sentinel-logstash-output {
      managed_identity => true
      tenant_id => "XXX"
	  data_collection_endpoint => "https://XXX2.uksouth-1.ingest.monitor.azure.com"
      dcr_immutable_id => "dcr-XXX2"
      dcr_stream_name => "Custom-XXX2"
    }
}

Question 1: Do I need two separate conf files? can it be combined into 1 file considering the logs are destined to different tables in Log Analytics Workspace?
Question 2: What causes the issue?

Error Message


[FATAL][org.logstash.Logstash    ] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
2

How are you running Logstash?

There are a couple of issues here, ports below 1024 are reserved and only a privileged user can bind to them, it is not recommended to run Logstash as root or as a privileged user, so it would be better to change the ports or configure your system to allow the logstash user to bind to those ports.

You can use only file, but would need to use conditionals, if you do not use conditionals the data received by all inputs will be sent to all outputs, also, even if you use 2 files, but do not configure Logstash to run multiple pipelines, those files would be merged into one configuration.

I recommend to use multiple files and configure the pipelines in pipelines.yml, as mentioned in this documentation.

You need to provide more context, the error you shared is not enough to troubleshoot the issue.

Please describe how you are running Logstash and share the full error you are getting.

3

Hi Leandro

Many thanks for your reply. I seem to find the issue with the pipelines.yml file where I used incorect syntax for the config file path. As it is running on Windows server, it needed double backslashes. I have copied the content below for the reference .

- pipeline.id: my_first_pipeline
  pipeline.workers: 1
  path.config: "C:\\logstash-8.17.1\\config\\myfirstconfigfile.conf"
- pipeline.id: my_second_pipeline
  pipeline.workers: 1
  path.config: "C:\\logstash-8.17.1\\config\\mysecondconfigfile.conf"