Can't not bind listen port 5140 to logstash ubuntu 18.04

Elastic Stack Logstash
1

Nothing is using that port,

guyp@ubuntu:/etc/logstash/conf.d$ netstat -an | grep 5140
tcp 0 0 127.0.0.1:9200 127.0.0.1:51406 ESTABLISHED
tcp 0 0 127.0.0.1:9200 127.0.0.1:51400 ESTABLISHED
tcp 0 0 127.0.0.1:51400 127.0.0.1:9200 ESTABLISHED
tcp 0 0 127.0.0.1:51408 127.0.0.1:9200 ESTABLISHED
tcp 0 0 127.0.0.1:9200 127.0.0.1:51408 ESTABLISHED
tcp 0 0 127.0.0.1:51406 127.0.0.1:9200 ESTABLISHED

guyp@ubuntu:/etc/logstash/conf.d$ grep udp /etc/logstash/conf.d/*
/etc/logstash/conf.d/01-inputs.conf: udp {
grep: /etc/logstash/conf.d/patterns: Is a directory

java version "12.0.2" 2019-07-16
Java(TM) SE Runtime Environment (build 12.0.2+10)
Java HotSpot(TM) 64-Bit Server VM (build 12.0.2+10, mixed mode, sharing)

curl: (7) Failed to connect to localhost port 9600: Connection refused
guyp@ubuntu:/etc/logstash/conf.d$ curl http://localhost:9200
{
"name" : "ubuntu",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "L4moMAVoRE-Vqi4NDBUOtw",
"version" : {
"number" : "7.4.0",
"build_flavor" : "default",
"build_type" : "deb",
"build_hash" : "22e1767283e61a198cb4db791ea66e3f11ab9910",
"build_date" : "2019-09-27T08:36:48.569419Z",
"build_snapshot" : false,
"lucene_version" : "8.2.0",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"

I have uninstalled/re-installed logstash and java w/ out success and at a loss.

Thanks

2

What is the complete configuration of the udp input and what error message do you get?

4

What error message do you get?

5

I assume because port 5140 is not open/listening....

Oct 5 11:27:38 kernel: arp: 192.168.1.116 moved from 74:d6:37:6c:7f:62 to 7c:61:66:14:cd:4e on xn1
Oct 5 11:27:35 syslog-ng[21178]: Syslog connection broken; fd='25', server='AF_INET(192.168.1.130:5140)', time_reopen='60'
Oct 5 11:27:35 syslog-ng[21178]: I/O error occurred while writing; fd='25', error='Connection refused (61)'
Oct 5 11:27:35 syslog-ng[21178]: Syslog connection established; fd='25', server='AF_INET(192.168.1.130:5140)', local='AF_INET(192.168.1.1:0)'
6

What is in the logstash logs?

7

[2019-10-05T12:30:40,460][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
[2019-10-05T12:30:45,509][INFO ][logstash.runner ] Logstash shut down.
[2019-10-05T12:30:58,925][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"7.4.0"}
[2019-10-05T12:30:59,919][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of #, input, filter, output at line 36, column 1 (byte 546) after ", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:41:in compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:49:incompile_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:11:in block in compile_sources'", "org/jruby/RubyArray.java:2584:inmap'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:10:in compile_sources'", "org/logstash/execution/AbstractPipelineExt.java:153:ininitialize'", "org/logstash/execution/JavaBasePipelineExt.java:47:in initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:26:ininitialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:36:in execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:326:inblock in converge_state'"]}
[2019-10-05T12:31:00,125][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
[2019-10-05T12:31:05,185][INFO ][logstash.runner ] Logstash shut down.

8

It looks like you have an error in one of your other config files which prevents Logstash from starting.

9

Yea, I have checked both 01 and 05 in /etc/logstash/conf.d, but cant determine the issue....

10

If you show all your config we may be able to help.

12 
# PF Firewall Logs
input {
  tcp {
    port => 5140
    type => "syslog"
  }
  udp {
    port => 5140
    type => "syslog"
  }
}

# 05-syslog.conf
filter {
  if [type] == "syslog" {
    #Adjust to match the IP address of pfSense or OPNSense
    if [host] =~ /192\.168\.1\.1/ {
      mutate {
        add_tag => ["pf", "Ready"]
      }
    }
     if [host] =~ /172\.2\.22\.1/ {
      mutate {
        add_tag => ["pf-2", "Ready"]
      }
    }
    if "Ready" not in [tags] {
      mutate {
        add_tag => [ "syslog" ]
      }
    }
  }
}
filter {
  if [type] == "syslog" {
    mutate {
      remove_tag => "Ready"
    }
  }
}
13

Is OPNsense on a separate line compared to the rest of the comment?

14

No, it's on the same line.

"ConfigurationError", :message=>"Expected one of #, => at line 25, column"

where/what config file is the above error referring to?

[2019-10-05T22:14:21,721][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
[2019-10-05T22:14:26,782][INFO ][logstash.runner ] Logstash shut down.
[2019-10-05T22:14:43,882][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"7.4.0"}
[2019-10-05T22:14:45,298][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of #, => at line 25, column 6 (byte 410) after filter {\n if [type] == "syslog" {\n #Adjust to match the IP address of pfSense or OPNSense\n if [host] =~ /192\.168\.1\.1/ {\n mutate {\n add_tag => ["pf", "Ready"]\n }\n if "Ready" not in [tags] {\n mutate {\n add_tag => [ "syslog" ]\n }\n }\n }\n}\nfilter {\n if ", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:41:in compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:49:incompile_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:11:in block in compile_sources'", "org/jruby/RubyArray.java:2584:inmap'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:10:in compile_sources'", "org/logstash/execution/AbstractPipelineExt.java:153:ininitialize'", "org/logstash/execution/JavaBasePipelineExt.java:47:in initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:26:ininitialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:36:in execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:326:inblock in converge_state'"]}
[2019-10-05T22:14:45,540][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
[2019-10-05T22:14:50,611][INFO ][logstash.runner ] Logstash shut down.

15

The log messages do not match the configuration you are showing us (the log does not include /172.2.22.1/). What configuration produced that log message?

17

You cannot nest a filter group inside a filter group.

filter {
    if [type] == "syslog" {
        #Adjust to match the IP address of pfSense or OPNSense
        if [host] =~ /192.168.1.1/ {
            mutate {
                add_tag => ["pf", "Ready"]
            }
            if "Ready" not in [tags] {
                mutate {
                    add_tag => [ "syslog" ]
                }
            }
        }
    }
    filter {
        if [type] == "syslog" {
            mutate {
                remove_tag => "Ready"
            }
        }
    }
18

So I would assume I need to remove the following from the above config?

filter {
if [type] == "syslog" {
mutate {
remove_tag => "Ready"
}
}
}

Thanks

19

You can leave the logic just remove that filter block / braces as you can not nest them as @Badger indicated but you can leave that if block / logic if you need to.

20

I am new to this so I appreciate the feedback/help, so is this correct:

# 05-syslog.conf
filter
if [type] == "syslog" {
    #Adjust to match the IP address of pfSense or OPNSense
    if [host] =~ /192\.168\.1\.1/ {
      mutate {
        add_tag => ["pf", "Ready"]
    }
    if "Ready" not in [tags] {
      mutate {
        add_tag => [ "syslog" ]
                }
            }
        }
    }
    if [type] == "syslog" {
            mutate {
                remove_tag => "Ready"
            }
        }
    }
21

It would really help if you formatted your code using the code formatter '</>' button on the post editor

22

Made the change and restarted logstash w/ same results:

guyp@ubuntu:/var/log/logstash$ tail logstash-plain.log
[2019-10-06T09:28:09,088][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
[2019-10-06T09:28:13,960][INFO ][logstash.runner ] Logstash shut down.
[2019-10-06T09:28:36,466][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"7.4.0"}
[2019-10-06T09:28:38,178][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of #, {, ,, ] at line 35, column 28 (byte 628) after filter {\n if "pf" in [tags] {\n grok {\n match => [ "message" ", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:41:in compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:49:incompile_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:11:in block in compile_sources'", "org/jruby/RubyArray.java:2584:inmap'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:10:in compile_sources'", "org/logstash/execution/AbstractPipelineExt.java:153:ininitialize'", "org/logstash/execution/JavaBasePipelineExt.java:47:in initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:26:ininitialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:36:in execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:326:inblock in converge_state'"]}
[2019-10-06T09:28:38,466][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
[2019-10-06T09:28:43,364][INFO ][logstash.runner ] Logstash shut down.
[2019-10-06T09:29:04,802][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"7.4.0"}
[2019-10-06T09:29:06,587][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of #, {, ,, ] at line 35, column 28 (byte 628) after filter {\n if "pf" in [tags] {\n grok {\n match => [ "message" ", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:41:in compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:49:incompile_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:11:in block in compile_sources'", "org/jruby/RubyArray.java:2584:inmap'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:10:in compile_sources'", "org/logstash/execution/AbstractPipelineExt.java:153:ininitialize'", "org/logstash/execution/JavaBasePipelineExt.java:47:in initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:26:ininitialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:36:in execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:326:inblock in converge_state'"]}
[2019-10-06T09:29:06,886][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
[2019-10-06T09:29:11,954][INFO ][logstash.runner ] Logstash shut down.

23 
05-syslog.conf
filter {
  if [type] == "syslog" {
    #Adjust to match the IP address of pfSense or OPNSense
    if [host] =~ /192\.168\.1\.1/ {
      mutate {
        add_tag => ["pf", "Ready"]
    }
    if "Ready" not in [tags] {
      mutate {
        add_tag => [ "syslog" ]
                }
            }
        }
    }
        if [type] == "syslog" {
            mutate {
                remove_tag => "Ready"
            }
        }
    }