Hi,
Configured rsyslog to send logs to logstash.
But kibana dosen't receive.
Please help.
OS : Ubuntu 20.04
</>
cat /etc/elasticsearch/elasticsearch.yml
Elasticsearch performs poorly when the system is swapping the memory.
---------------------------------- Network -----------------------------------
Set the bind address to a specific IP (IPv4 or IPv6):
#network.host: localhost
network.host: 172.20.111.199
Set a custom port for HTTP:
http.port: 9200
For more information, consult the network module documentation.
cat /etc/logstash/conf.d/02-beats-input.conf
This input block will listen on port 10514 for logs to come in.
host should be an IP on the Logstash server.
codec => "json" indicates that we expect the lines we're receiving to be in JSON format
type => "rsyslog" is an optional identifier to help identify messaging streams in the pipeline.
input {
udp {
host => "172.20.111.199"
port => 10514
codec => "json"
type => "rsyslog"
}
}
# cat /etc/logstash/conf.d/30-elasticsearch-output.conf
This is an empty filter block. You can later add other filters here to further process
your log lines
filter { }
This output block will send all events of type "rsyslog" to Elasticsearch at the configured
host and port into daily indices of the pattern, "rsyslog-YYYY.MM.DD"
output {
if [type] == "rsyslog" {
elasticsearch {
hosts => [ "172.20.111.199:9200" ]
}
}
}
cat /etc/filebeat/filebeat.yml
------------------------------ Logstash Output -------------------------------
output.logstash:
The Logstash hosts
#hosts: ["172.20.111.199:10514"]
hosts: ["172.20.111.199:5044"]
cat /etc/kibana/kibana.yml
The URLs of the Elasticsearch instances to use for all your queries.
elasticsearch.hosts: ["http://172.20.111.199:9200"]
#elasticsearch.hosts: ["http://localhost:9200"]
root@elk:/etc/rsyslog.d# ls
01-json-template.conf 20-ufw.conf 21-cloudinit.conf 50-default.conf 60-output.conf
# cat 01-json-template.conf
template(name="json-template"
type="list") {
constant(value="{")
constant(value=""@timestamp":"") property(name="timereported" dateFormat="rfc3339")
constant(value="","@version":"1")
constant(value="","message":"") property(name="msg" format="json")
constant(value="","sysloghost":"") property(name="hostname")
constant(value="","severity":"") property(name="syslogseverity-text")
constant(value="","facility":"") property(name="syslogfacility-text")
constant(value="","programname":"") property(name="programname")
constant(value="","procid":"") property(name="procid")
constant(value=""}\n")
}
# cat 60-output.conf
This line sends all lines to defined IP address at port 10514,
using the "json-template" format template
. @172.20.111.199:10514;json-template
cat /var/log/syslog
Feb 21 11:04:02 elk filebeat[4471]: 2021-02-21T11:04:02.296+0530#011ERROR#011[publisher_pipeline_output]#011pipeline/output.go:154#011Failed to connect to backoff(async(tcp://172.20.111.199:5044)): dial tcp 172.20.111.199:5044: connect: connection refused
Feb 21 11:04:02 elk filebeat[4471]: 2021-02-21T11:04:02.296+0530#011INFO#011[publisher_pipeline_output]#011pipeline/output.go:145#011Attempting to reconnect to backoff(async(tcp://172.20.111.199:5044)) with 1168 reconnect attempt(s)
indent preformatted text by 4 spaces
Kibana didnt received any log.
Please help to rectify this issue.
