Logstash-forwarder documentaion

Elastic Stack Logstash
1

Hi Team,

Is there any document for setting up logstash-forwarder ?

2

Well, there's the project's README file, which should include most things you need to know. That file also tells you that logstash-forwarder is deprecated and replaced by Filebeat. So, use Filebeat instead.

4

At this time i am going with logstash-forwarder and will move to filefeat later. I have few queries can you help?

  1. If I am right logstash-forwarder will push the log to the ELK hosted server. What would be the path in ELK server where the logs will be pushed from logstash-forwarder. As I understand that we need to configure the log path with grok filter in logstash.conf

  2. What is the difference and relation among Filebeat, Libbeat, Packetbeat and Topbeat?

5
  1. logstash-forwarder will ship log events over the network directly to a Logstash instance, typically on another server. Unless you explicitly configure Logstash to stream those logs on an on-disk file they will never be stored on disk.
  2. Filebeat, Packetbeat, and Topbeat collect and ship different kinds of data and they all use libbeat as a library with common functionality.
6

ok, then what would be the path in input section. I am little bit confused here.

input {
file {
type => mobile_messages
path => [ "/installdir/ELK/LOGS/MOBILE/messages.log" ]
start_position => "beginning"
}
}

filter {

if [type] == "mobile_messages" {
grok {
type => "mobile_messages"

match => [ "message", "%{GREEDYDATA:server_name}; \/%{WORD:country_name}; %{DATESTAMP:datestamp}; %{WORD:LOB}; %{GREEDYDATA:gd1} (?:\[%{WORD:service_name}\]) %{GREEDYDATA:gd2}(?:\<StatusCode\>%{NUMBER:statuscode}\<\/StatusCode\>)(?:\<Severity\>%{WORD:severity}\<\/Severity\>)(?:\<StatusDesc\>%{GREEDYDATA:statusdesc}\<\/StatusDesc\>)%{GREEDYDATA:gd3}" ]

}

}
if "_grokparsefailure" in [tags] { drop {} }
}

output {
elasticsearch {
cluster => "elasticsearch"
protocol => "http"
action => "index"
host => "100.x.x.x"
index => "mobile_messages"
workers => 1
}
stdout {
codec => json
}
}

7

Don't use a file input, use a lumberjack input (or a beats input if using filebeat). See https://github.com/elastic/logstash-forwarder#use-with-logstash.