Unable to enable port 5044 for logstash

Quan_Chu · March 15, 2024, 3:11am

here is my config:
input {
beats {
port => 5044
}
}

filter {
if [fileset][module] == "system" {
if [fileset][name] == "auth" {
grok {
match => { "message" => ["%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} sshd(?:[%{POSINT:[system][auth][pid]}])?: %{DATA:[system][auth][ssh][event]} %{DATA:[system][auth][ssh][method]} for (invalid user )?%{DATA:[system][auth][user]} from %{IPORHOST:[system][auth][ssh][ip]} port %{NUMBER:[system][auth][ssh][port]} ssh2(: %{GREEDYDATA:[system][auth][ssh][signature]})?",
"%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} sshd(?:[%{POSINT:[system][auth][pid]}])?: %{DATA:[system][auth][ssh][event]} user %{DATA:[system][auth][user]} from %{IPORHOST:[system][auth][ssh][ip]}",
"%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} sshd(?:[%{POSINT:[system][auth][pid]}])?: Did not receive identification string from %{IPORHOST:[system][auth][ssh][dropped_ip]}",
"%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} sudo(?:[%{POSINT:[system][auth][pid]}])?: \s*%{DATA:[system][auth][user]} %{DATA:[system][auth][sudo][error]} ;)? TTY=%{DATA:[system][auth][sudo][tty]} ; PWD=%{DATA:[system][auth][sudo][pwd]} ; USER=%{DATA:[system][auth][sudo][user]} ; COMMAND=%{GREEDYDATA:[system][auth][sudo][command]}",
"%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} groupadd(?:[%{POSINT:[system][auth][pid]}])?: new group: name=%{DATA:system.auth.groupadd.name}, GID=%{NUMBER:system.auth.groupadd.gid}",
"%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} useradd(?:[%{POSINT:[system][auth][pid]}])?: new user: name=%{DATA:[system][auth][user][add][name]}, UID=%{NUMBER:[system][auth][user][add][uid]}, GID=%{NUMBER:[system][auth][user][add][gid]}, home=%{DATA:[system][auth][user][add][home]}, shell=%{DATA:[system][auth][user][add][shell]}$",
"%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} %{DATA:[system][auth][program]}(?:[%{POSINT:[system][auth][pid]}])?: %{GREEDYMULTILINE:[system][auth][message]}"] }
pattern_definitions => {
"GREEDYMULTILINE"=> "(.|\n)"
}
remove_field => "message"
}
date {
match => [ "[system][auth][timestamp]", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
geoip {
source => "[system][auth][ssh][ip]"
target => "[system][auth][ssh][geoip]"
}
}
else if [fileset][name] == "syslog" {
grok {
match => { "message" => ["%{SYSLOGTIMESTAMP:[system][syslog][timestamp]} %{SYSLOGHOST:[system][syslog][hostname]} %{DATA:[system][syslog][program]}(?:[%{POSINT:[system][syslog][pid]}])?: %{GREEDYMULTILINE:[system][syslog][message]}"] }
pattern_definitions => { "GREEDYMULTILINE" => "(.|\n)" }
remove_field => "message"
}
date {
match => [ "[system][syslog][timestamp]", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}
}

output {
elasticsearch {
hosts => "localhost:9200"
manage_template => false
index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
}
}

the service is running:
[root@localhost ~]# systemctl status logstash
● logstash.service - logstash
Loaded: loaded (/etc/systemd/system/logstash.service; enabled; vendor preset: disabled)
Active: active (running) since Thu 2024-03-14 23:04:59 EDT; 6s ago
Main PID: 125716 (java)
Tasks: 15 (limit: 22840)
Memory: 333.3M
CGroup: /system.slice/logstash.service
└─125716 /usr/share/logstash/jdk/bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Djava.awt.headless=true -Dfile.encoding=>

Mar 14 23:04:59 localhost.localdomain systemd[1]: Started logstash.
Mar 14 23:04:59 localhost.localdomain logstash[125716]: Using bundled JDK: /usr/share/logstash/jdk
Mar 14 23:04:59 localhost.localdomain logstash[125716]: OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.

but port 5044 was not enabled.
[root@localhost ~]# netstat -nutlp
Active Internet connections Proto Recv-Q Send-Q Local Address tcp 0 0 127.0.0.1:631 tcp 0 0 0.0.0.0:111 tcp 0 0 0.0.0.0:22 tcp 0 0 0.0.0.0:5601 tcp 0 0 127.0.0.1:6010 tcp 0 0 192.168.122.1:53 tcp6 0 0 ::1:631 tcp6 0 0 127.0.0.1:9200 tcp6 0 0 :::111 tcp6 0 0 ::1:9200 tcp6 0 0 :::22 tcp6 0 0 ::1:6010 tcp6 0 0 ::1:9300 tcp6 0 0 127.0.0.1:9300 udp 0 0 0.0.0.0:111 udp 0 0 127.0.0.1:323 udp 0 0 0.0.0.0:5353 udp 0 0 0.0.0.0:57215 udp 0 0 192.168.122.1:53 udp 0 0 0.0.0.0:67 udp6 0 0 :::111 udp6 0 0 ::1:323 udp6 0 0 :::5353 udp6 0 0 :::53299 (only servers)
Foreign Address State PID/Program name
0.0.0.0:* LISTEN 1057/cupsd
0.0.0.0:* LISTEN 1/systemd
0.0.0.0:* LISTEN 1056/sshd
0.0.0.0:* LISTEN 1433/node
0.0.0.0:* LISTEN 71909/sshd: devops@
0.0.0.0:* LISTEN 2093/dnsmasq
:::* LISTEN 1057/cupsd
:::* LISTEN 1432/java
:::* LISTEN 1/systemd
:::* LISTEN 1432/java
:::* LISTEN 1056/sshd
:::* LISTEN 71909/sshd: devops@
:::* LISTEN 1432/java
:::* LISTEN 1432/java
0.0.0.0:* 1/systemd
0.0.0.0:* 919/chronyd
0.0.0.0:* 902/avahi-daemon: r
0.0.0.0:* 902/avahi-daemon: r
0.0.0.0:* 2093/dnsmasq
0.0.0.0:* 2093/dnsmasq
:::* 1/systemd
:::* 919/chronyd
:::* 902/avahi-daemon: r
:::* 902/avahi-daemon: r

Please help!

Rios · March 15, 2024, 5:03am

I have tested your .conf, it looks OK.

Check the /var/log/logstash/logstash-plain.log file, there should be more info.

Quan_Chu · March 15, 2024, 6:22am

here is the info from logstash-plain.log file:
[2024-03-14T23:02:44,046][INFO ][org.reflections.Reflections] Reflections took 73 ms to scan 1 urls, producing 119 keys and 419 values
[2024-03-14T23:02:44,786][INFO ][logstash.runner ] Using config.test_and_exit mode. Config Validation Result: OK. Exiting Logstash
[2024-03-14T23:04:21,455][INFO ][logstash.runner ] Log4j configuration path used is: /etc/logstash/log4j2.properties
[2024-03-14T23:04:21,483][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"7.17.18", "jruby.version"=>"jruby 9.2.20.1 (2.5.8) 2021-11-30 2a2962fbd1 OpenJDK 64-Bit Server VM 11.0.20+8 on 11.0.20+8 +indy +jit [linux-x86_64]"}
[2024-03-14T23:04:21,484][INFO ][logstash.runner ] JVM bootstrap flags: [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djdk.io.File.enableADS=true, -Djruby.compile.invokedynamic=true, -Djruby.jit.threshold=0, -Djruby.regexp.interruptible=true, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true]
[2024-03-14T23:04:23,211][INFO ][org.reflections.Reflections] Reflections took 54 ms to scan 1 urls, producing 119 keys and 419 values
[2024-03-14T23:04:24,003][INFO ][logstash.runner ] Using config.test_and_exit mode. Config Validation Result: OK. Exiting Logstash

Rios · March 15, 2024, 7:32pm

That is only -t, test/validation .conf file. Should be more. If there is no newer records, change ownership to "logstash" user for /var/log/logstash/ and files.

system · April 12, 2024, 7:32pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.