Metricbeat Communication problem

Elastic Stack Logstash
1

Hello,

Tearing my hair out at this early juncture.

I have setup Logstash/Elastisearch/Kibana on a Redhat Server and I have setup MetricBeats on my Windows machine.

Getting the following in the metricbeat logs:
2018-03-28T11:54:12.901+0100 ERROR pipeline/output.go:74 Failed to connect: dial tcp 192.168.212.5:5044: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
2018-03-28T11:54:26.802+0100 INFO [monitoring] log/log.go:124 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":8156,"time":8156},"total":{"ticks":10890,"time":10890,"value":10890},"user":{"ticks":2734,"time":2734}},"info":{"ephemeral_id":"f65bdd7b-6ff3-4d85-badb-5fb97861d575","uptime":{"ms":691152}},"memstats":{"gc_next":12256944,"memory_alloc":10589952,"memory_total":161017392,"rss":45056}},"libbeat":{"config":{"module":{"running":3}},"pipeline":{"clients":6,"events":{"active":1096,"published":51,"retry":13,"total":51}}},"metricbeat":{"system":{"cpu":{"events":3,"success":3},"filesystem":{"events":3,"success":3},"fsstat":{"events":1,"success":1},"memory":{"events":3,"success":3},"network":{"events":15,"success":15},"process":{"events":23,"success":23},"process_summary":{"events":3,"success":3}}}}}}

Non commented lines in config are as follows:
[itsuser@elk tmp]$ sudo grep -v '#' /etc/logstash/logstash.yml
path.data: /var/lib/logstash

http.host: "192.168.212.5"
http.port: 9600-9700
path.logs: /var/log/logstash

This is what is logged:
[itsuser@elk tmp]$ sudo tail /var/log/logstash/logstash-plain.log
[2018-03-28T12:08:43,202][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"6.2.3"}
[2018-03-28T12:08:43,287][INFO ][logstash.config.source.local.configpathloader] No config files found in path {:path=>"/etc/logstash/conf.d/.conf"}
[2018-03-28T12:08:43,288][ERROR][logstash.config.sourceloader] No configuration found in the configured sources.
[2018-03-28T12:08:43,382][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
[2018-03-28T12:09:29,788][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"fb_apache", :directory=>"/usr/share/logstash/modules/fb_apache/configuration"}
[2018-03-28T12:09:29,795][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"netflow", :directory=>"/usr/share/logstash/modules/netflow/configuration"}
[2018-03-28T12:09:30,873][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"6.2.3"}
[2018-03-28T12:09:30,992][INFO ][logstash.config.source.local.configpathloader] No config files found in path {:path=>"/etc/logstash/conf.d/.conf"}
[2018-03-28T12:09:30,993][ERROR][logstash.config.sourceloader] No configuration found in the configured sources.
[2018-03-28T12:09:31,106][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}

With a "netstat -an|grep 5044" also I can't see anything listening on that port.

Am I missing options from the config?

Regards,
David

2

It seems like a Logstash problem based on the missing listener on port 5044.
Please post your question in the Logstash forum.

3

Hello,

Was advised to move this here as logstash doesn't seem to be listening.

Seeing following in logs and not sure is this to do with java permissions:
[2018-03-28T16:53:21,561][FATAL][logstash.runner ] An unexpected error occurred! {:error=>#<SocketError: initialize: name or service not known>, :backtrace=>["org/jruby/ext/socket/RubyTCPServer.java:124:in initialize'", "org/jruby/RubyIO.java:875:innew'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/puma-2.16.0-java/lib/puma/binder.rb:234:in add_tcp_listener'", "(eval):2:inadd_tcp_listener'", "/usr/share/logstash/logstash-core/lib/logstash/webserver.rb:88:in start_webserver'", "/usr/share/logstash/logstash-core/lib/logstash/webserver.rb:44:inblock in run'", "org/jruby/RubyRange.java:485:in each'", "org/jruby/RubyEnumerable.java:1067:ineach_with_index'", "/usr/share/logstash/logstash-core/lib/logstash/webserver.rb:39:in run'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:374:inblock in start_webserver'"]}
[2018-03-28T16:53:21,599][ERROR][org.logstash.Logstash ] java.lang.IllegalStateException: org.jruby.exceptions.RaiseException: (SystemExit) exit

Hopefully someone can offer some advice on where I'm going wrong.

4

Have you created any Logstash config file that has a beats input listening to port 5044? If so, what does it look like?

5

Yes I have. Here you go. Thanks.

input {
beats {
port => 5044
}
}

output {
elasticsearch {
hosts => "localhost:9200"
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}

Actually should have added noticed my mistake with the missing conf file but getting the error above now.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.