Filebeat -getsockopt: connection refused

Elastic Stack Beats
1

Hi,

Here my filebeat.yml

filebeat.prospectors:
- input_type: log
  document_type: web_log
  paths:
    - /var/log/web.log
  fields:
    log_type: weblog
  fields_under_root: true

- input_type: log
  document_type: email_log
  paths:
    - /var/log/email.log
  fields:
    log_type: emaillog
  fields_under_root: true
name: 10.0.0.0
output.logstash:
  # The Logstash hosts
  #hosts: ["localhost:5044"]
  hosts: ["10.141.127.145:5443"]
  bulk_max_size: 2048
  #ssl.certificate_authorities: ["/etc/filebeat/logstash.crt"]
  template.name: "filebeat"
  template.path: "filebeat.template.json"
  template.overwrite: false

While am restarting service throwing error as

ERR Connecting error publishing events (retrying): dial tcp 10.141.127.145:5443: getsockopt: connection refused

Telnet to 10.141.127.145:5443is not working from filebeat, my ELK is running on 10.141.127.145 server, where my conf.d/ files as below

#input.conf

input {
  beats {
    host => "0.0.0.0"
    port => 5443
    type => weblog
    #ssl => true
    #ssl_certificate => "/etc/logstash/logstash.crt"
    #ssl_key => "/etc/logstash/logstash.key"
  }
}

#filter.conf

filter{
  if [type] == "weblog" {
    grok {
      match => [ "message", "%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:loglevel} (?<logger>(?:[a-zA-Z0-9]+\.)*[-A-Za-z0-9$]+) %{GREEDYDATA:message}"]
      overwrite => [ "message" ]
    }

  }
}

#output.conf

output {
  elasticsearch { hosts => ["10.141.127.145:9200"]
    hosts => "10.141.127.145:9200"
    manage_template => false
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
    document_type => "%{[@metadata][type]}"
  }
}

Noticed process running under "logstash" user account instead of root.. see below

root@elk-ubuntu:/usr/share/logstash# ps -eaf | grep logstash
logstash  3364     1 16 18:26 ?        00:00:36 /usr/bin/java -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+DisableExplicitGC -Djava.awt.headless=true -Dfile.encoding=UTF-8 -XX:+HeapDumpOnOutOfMemoryError -Xmx1g -Xms256m -Xss2048k -Djffi.boot.library.path=/usr/share/logstash/vendor/jruby/lib/jni -Xbootclasspath/a:/usr/share/logstash/vendor/jruby/lib/jruby.jar -classpath : -Djruby.home=/usr/share/logstash/vendor/jruby -Djruby.lib=/usr/share/logstash/vendor/jruby/lib -Djruby.script=jruby -Djruby.shell=/bin/sh org.jruby.Main /usr/share/logstash/lib/bootstrap/environment.rb logstash/runner.rb --path.settings /etc/logstash
root      3402  2485  0 18:29 pts/6    00:00:00 grep --color=auto logstash

I have restarted logstash stop & kill, but in ELK i couldn't see listening port on 5443 using netstat command, and filbeat not able to connect. May I know how to fix it.

No firewall issue, all ports are opened between ELK & Filebeat and it was working earlier.. i have restarted server thats it.

Thanks :slight_smile:

2

connection refused indicates the port not being available. For example if Logstash is down.

Check your logstash logs. In case logstash is not startup up correctly, try to run it in debug mode.

In you Logstash output section, hosts is defined twice. Maybe it's a config file parsing error?

3

Found it was because of restarting logstash server taking too long time and no logs captured in logstash.plain.log, May I know how to increase performance tuning and find which is slowing down.. I have 16 GB RAM/ 100 GB for ELK server

4

Just noticed logstash log as below

root@elk-ubuntu:/etc/logstash/conf.d# tail -f /var/log/logstash/logstash-plain.log
[2017-07-04T11:08:18,113][WARN ][logstash.runner          ] SIGTERM received. Shutting down the agent.
[2017-07-04T11:08:18,136][WARN ][logstash.agent           ] stopping pipeline {:id=>"main"}
[2017-07-04T11:10:11,120][ERROR][logstash.agent           ] Cannot create pipeline {:reason=>"Expected one of #, {, ,, ] at line 4, column 26 (byte 72) after filter{\n  #if [type] == \"weblog\" {\n    grok {\n      match => [ message "}

Port 5443 will start after sometime.. though this error

5

seems like Logstash is complaining about syntax in your configuration files.

6

I have given my logstash input/output/filter above, please let me know where is syntax error

7

I'm no compiler and I don't use Logstash that often that I can tell what's the actual error.

Reading the error message: [2017-07-04T11:10:11,120][ERROR][logstash.agent ] Cannot create pipeline {:reason=>"Expected one of #, {, ,, ] at line 4, column 26 (byte 72) after filter{\n #if [type] == \"weblog\" {\n grok {\n match => [ message "}

my guess it's in filter.conf in the match setting. I think it should be a dictionary like:

match => {
  "message" => "..."
}
8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.