Unable to send logs to remote Logstash server. working fine in localhost

Elastic Stack Logstash
1

Hi, i am new to Elasticstack i configure filebeat in client machine, to send logs to logstash- it is not sending logs to logstash server. Here when I locally send logs to logstash server using filebeat its working fine.

Any help would be appreciated , Thanks

2

Welcome to our community! :smiley:

What do your config files look like?

3

HI warkolm, Thanks for kind update, below is my config files details.

I have one elk server running with ( logstash, Kibana, Elasticsearch). I have to monitor logs from remote apache server. i have configure filebeat in apache server and it is running fine.

filebeat.yml file at client side:

  • type: log
    enabled: true
    paths:
    • /var/log/apache2/*.log

filebeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: false
setup.template.settings:
index.number_of_shards: 1
output.logstash:
hosts: ["Remote-ELkServer:5044"]

02-beats-input.conf in ELK (/etc/logstash/conf.d):
input {
beats {
port => 5044
host => "0.0.0.0"
}
}

10-apache2-filter.conf at ELK server:

filter {
if [fileset][module] == "apache2" {
if [fileset][name] == "access" {
grok {
match => { "message" => ["%{IPORHOST:[apache2][access][remote_ip]} - %{DATA:[apache2][access][user_name]} [%{HTTPDATE:[apache2][access
][time]}] "%{WORD:[apache2][access][method]} %{DATA:[apache2][access][url]} HTTP/%{NUMBER:[apache2][access][http_version]}" %{NUMBER:[apache
2][access][response_code]} %{NUMBER:[apache2][access][body_sent][bytes]}( "%{DATA:[apache2][access][referrer]}")?( "%{DATA:[apache2][access]
[agent]}")?",
"%{IPORHOST:[apache2][access][remote_ip]} - %{DATA:[apache2][access][user_name]} \[%{HTTPDATE:[apache2][access][time]}\] "-" %{NU
MBER:[apache2][access][response_code]} -" ] }
remove_field => "message"
}
mutate {
add_field => { "read_timestamp" => "%{@timestamp}" }
}
date {
match => [ "[apache2][access][time]", "dd/MMM/YYYY:H:m:s Z" ]
remove_field => "[apache2][access][time]"
}
useragent {
source => "[apache2][access][agent]"
target => "[apache2][access][user_agent]"
remove_field => "[apache2][access][agent]"
}
geoip {
source => "[apache2][access][remote_ip]"
target => "[apache2][access][geoip]"
}
}
else if [fileset][name] == "error" {
grok {
match => { "message" => ["[%{APACHE_TIME:[apache2][error][timestamp]}] [%{LOGLEVEL:[apache2][error][level]}]( [client %{IPORHOST:[
apache2][error][client]}])? %{GREEDYDATA:[apache2][error][message]}",
"[%{APACHE_TIME:[apache2][error][timestamp]}] [%{DATA:[apache2][error][module]}:%{LOGLEVEL:[apache2][error][level]}] [pid %{NUMB
ER:[apache2][error][pid]}(:tid %{NUMBER:[apache2][error][tid]})?]( [client %{IPORHOST:[apache2][error][client]}])? %{GREEDYDATA:[apache2][er
ror][message1]}" ] }
pattern_definitions => {
"APACHE_TIME" => "%{DAY} %{MONTH} %{MONTHDAY} %{TIME} %{YEAR}"
}
remove_field => "message"
}
mutate {
rename => { "[apache2][error][message1]" => "[apache2][error][message]" }
}
date {
match => [ "[apache2][error][timestamp]", "EEE MMM dd H:m:s YYYY", "EEE MMM dd H:m:s.SSSSSS YYYY" ]
remove_field => "[apache2][error][timestamp]"
}
}
}
}

30-apache2-elasticserchoutput.conf at ELK server:
output {
elasticsearch {
hosts => localhost
manage_template => false
index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
}
}

Can you please clear my doubt.. do i have to configure logstash in client side also. ?
at client side i just configure filebeat and mention the paths.
I have also enable apache modue in ELK server.

filebeat test output from client machine.

logstash: 192.168.XX.XX:5044...
connection...
parse host... OK
dns lookup... OK
addresses: 192.168.XX.XX
dial up... OK
TLS... WARN secure connection disabled
talk to server... OK

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.