Logstash & Filebeat sending file/logs over

[skyluke.1987 (Luke)]

Hi all, I have this setup. Elasticsearch was setup on Server A with filebeat. Next I have Server B setup with logstash.

Is it correct that I send Nginx logs from Server B using "logstash" to server A? and Filebeat on server A will pickup and stores it in Elasticsearch.

Kindly point out which part I did wrongly. Thanks

[faec (Fae Charlton)]

Hi! It sounds like you have it backwards: Filebeat can send to Logstash and / or Elasticsearch, but Logstash cannot send to Filebeat. If you install Filebeat on the same server as your nginx logs, you can forward them directly to Elasticsearch, or run them through Logstash first, but once they're in Logstash they don't need to go back to Filebeat.

[skyluke.1987 (Luke)]

Hi faec, thanks for your sharing. I would like to ask if I can use logstash directly to get input data from another server B?

But when I do so, the system prompt me for absolute path of the data. How can I achieve that ?

Secondly, my (server B) which is a CentOS server. When I installed the Logstash on it, I found the settings to be strange. The logstash.conf file has been located at /etc/init/logstash.conf. May I know should I move it back to /usr/share/logstash?

[faec (Fae Charlton)]

Generally to get data off a server you will have to install something on it (logstash or a beat), or explicitly configure a service to forward the data in cases where it's supported (e.g. syslog).

The location of the logstash configuration may depend on how you installed it or other context... for more details try asking in https://discuss.elastic.co/c/logstash

[skyluke.1987 (Luke)]

Hi Faec,

I notice there is an error "[2019-12-09T10:52:46,692][ERROR][org.logstash.Logstash ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit" on the Logstash's log file. Is it due to the JAVA's runtime error? I'm currently using java version 1.8.0_131.

[2019-12-09T11:51:23,955][FATAL][logstash.runner ] An unexpected error occurred! {:error=>#<Errno::EADDRNOTAVAIL: Cannot assign requested address - bind - Cannot assign requested address>, :backtrace=>["org/jruby/ext/socket/RubyTCPServer.java:127:in initialize'", "org/jruby/RubyIO.java:876:in new'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/puma-2.16.0-java/lib/puma/binder.rb:234:in add_tcp_listener'", "(eval):2:in add_tcp_listener'", "/usr/share/logstash/logstash-core/lib/logstash/webserver.rb:88:in start_webserver'", "/usr/share/logstash/logstash-core/lib/logstash/webserver.rb:44:in block in run'", "org/jruby/RubyRange.java:511:in each'", "org/jruby/RubyEnumerable.java:1194:in each_with_index'", "/usr/share/logstash/logstash-core/lib/logstash/webserver.rb:39:in run'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:376:in block in start_webserver'"]}

Kindly share what can I do to help resolve this issue. Thanks