Syslog_severity_code it's always "5" , syslog_facility it's always "1", syslog_facility it's always "user-level" and syslog_severity it's always "notice"

Hello ELK's Friends

I am trying to set up an ELK system to replace an Rsyslog + Adiscon LogAnalyzer implementation. The operating system used was CentOS 7.3 The versions of the installed packages were the following:

filebeat.x86_64 0:5.4.1-1

kibana.x86_64 0:5.4.1-1

logstash.noarch 1:5.4.1-1

elasticsearch.noarch 0:5.4.1-1

The Logstash configuration files are as follows:

/etc/logstash/conf.d/input.conf

input {
beats {
port => 5044
ssl => true
ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
}
}

/etc/logstash/conf.d/output.conf

output {
elasticsearch {
hosts => ["localhost:9200"]
sniffing => true
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}

/etc/logstash/conf.d/filter.conf

filter {
if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:[%{POSINT:syslog_pid}])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
}
syslog_pri { }
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}

In the client operating system (CentOS 7.3)

FileBeat was configured as follows:

(No comment)

/etc/filebeat/filebeat.yml

filebeat:
prospectors:
-
paths:
- /var/log/*.log
- /var/log/secure
- /var/log/dmesg
- /var/log/messages
input_type: log
document_type: syslog
registry_file: /var/lib/filebeat/registry
output:
elasticsearch:
hosts: ["localhost:9200"]
logstash:
hosts: ["elk.example.com:5044"]
tls:
certificate_authorities: ["/etc/pki/tls/certs/logstash-forwarder.crt"]
shipper:
logging:
files:

But in Kibana, all events are classified as well

syslog_severity_code 5 or -

syslog_facility user-level or -

Any help will be greatly appreciated.

Hi
Can you make paths: section entry one line like

paths:
- /var/log/messages

Also comment/remove the following logstash hosts section.

logstash:
hosts: ["elk.example.com:5044"]

Restart the logstash and run the following in the terminal -

$ tail -f /var/log/messages

Now check whether all the logs that have been shown by above command are transferred to kibana dashboard.

Regards

Hi Makra

But if I remove the hosts section, How can I found the S.O Server, from the S.O Client ?

Hi
Julian
I assume your logstash host is remote one where you are sending the logs directly (port 5044) and ES is localhost. Now, Do you want the logs to be transferred to ES and logstash host simultaneously ?
It looks to me your configuration is correct. What is the result of tail command ? Does the same logs appear in the Kibana ? If that is the case then you need check the syslog configuration.

The syslog_pri filter reads a field with the encoded facility and severity information. By default it reads the syslog_pri field. Do you have a syslog_pri field?

Yeah that's the idea Makra simultaneously. Tail a which archive Makra ?

Hi magnusbaeck

Please look at this:

/etc/logstash/conf.d/filter.conf

filter {
if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:[%{POSINT:syslog_pid}])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
}
syslog_pri { }
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}

Add log files one by one. Lets tail syslog first and check that the logs do arrive in the kibana. If you have something wrong in logstash filter configuration then there will be some events that you can see in the output of tail command but will not be there in kibana. This will make sure that the logstash configuration is correct.

Make filebeat paths: section points to /var/log/syslog

paths:
- /var/log/messages
.................................................

Check the syslog that arrives in the files.

$ tail -f /var/log/syslog

Hope this helps

Mmmm my setup is like this elkserver.example.com (That server hosts Elesticsearch, Logstash and Kibana) and elkclient (Only has filebeat)

In elkclient.example.com with only /var/log/messages path

tail -f /var/log/messages

Jun 13 11:31:25 elkclient systemd: Time has been changed
Jun 13 11:31:25 elkclient chronyd[813]: System clock was stepped by 14401.230573 seconds
Jun 13 11:31:32 elkclient systemd: Created slice user-1000.slice.
Jun 13 11:31:32 elkclient systemd: Starting user-1000.slice.
Jun 13 11:31:32 elkclient systemd: Started Session 2 of user cnscadmin.
Jun 13 11:31:32 elkclient systemd-logind: New session 2 of user cnscadmin.
Jun 13 11:31:32 elkclient systemd: Starting Session 2 of user cnscadmin.
Jun 13 11:31:33 elkclient bash[1300]: cnscadmin [ 192.168.7.126 ] : #011 su -
Jun 13 11:31:37 elkclient su: (to root) cnscadmin on pts/0
Jun 13 11:31:38 elkclient bash[1359]: root [ ] : #011 systemctl reboot

In elkserver.example.com

curl -XGET 'http://localhost:9200/filebeat-*/_search?pretty':

{
"took" : 3,
"timed_out" : false,
"_shards" : {
"total" : 50,
"successful" : 50,
"failed" : 0
},
"hits" : {
"total" : 42218,
"max_score" : 1.0,
"hits" : [
{
"_index" : "filebeat-2017.05.25",
"_type" : "log",
"_id" : "AVxBPPh4yM45-hZBb-Ff",
"_score" : 1.0,
"_source" : {
"@timestamp" : "2017-05-25T20:12:08.669Z",
"beat" : {
"hostname" : "elk.example.com",
"name" : "elk.example.com",
"version" : "5.4.0"
},
"input_type" : "log",
"message" : "Aug 01 10:40:01 Updated: systemd-libs-219-19.el7_2.11.x86_64",
"offset" : 300,
"source" : "/var/log/yum.log",
"type" : "log"
}
},
{
"_index" : "filebeat-2017.05.25",
"_type" : "log",
"_id" : "AVxBPPh4yM45-hZBb-Fh",
"_score" : 1.0,
"_source" : {
"@timestamp" : "2017-05-25T20:12:08.669Z",
"beat" : {
"hostname" : "elk.example.com",
"name" : "elk.example.com",
"version" : "5.4.0"
},
"input_type" : "log",
"message" : "Aug 01 10:40:01 Updated: nss-util-3.21.0-2.2.el7_2.x86_64",
"offset" : 410,
"source" : "/var/log/yum.log",
"type" : "log"
}
},
{
"_index" : "filebeat-2017.05.25",
"_type" : "log",
"_id" : "AVxBPPh4yM45-hZBb-Fj",
"_score" : 1.0,
"_source" : {
"@timestamp" : "2017-05-25T20:12:08.669Z",
"beat" : {
"hostname" : "elk.example.com",
"name" : "elk.example.com",
"version" : "5.4.0"
},
"input_type" : "log",
"message" : "Aug 01 10:40:01 Updated: libgudev1-219-19.el7_2.11.x86_64",
"offset" : 526,
"source" : "/var/log/yum.log",
"type" : "log"
}
},
{
"_index" : "filebeat-2017.05.25",
"_type" : "log",
"_id" : "AVxBPPh4yM45-hZBb-Fk",
"_score" : 1.0,
"_source" : {
"@timestamp" : "2017-05-25T20:12:08.669Z",
"beat" : {
"hostname" : "elk.example.com",
"name" : "elk.example.com",
"version" : "5.4.0"
},
"input_type" : "log",
"message" : "[\u001B[32m OK \u001B[0m] Started Show Plymouth Boot Screen.",
"offset" : 54,
"source" : "/var/log/boot.log",
"type" : "log"
}
},
{
"_index" : "filebeat-2017.05.25",
"_type" : "log",
"_id" : "AVxBPPh4yM45-hZBb-Fm",
"_score" : 1.0,
"_source" : {
"@timestamp" : "2017-05-25T20:12:08.669Z",
"beat" : {
"hostname" : "elk.example.com",
"name" : "elk.example.com",
"version" : "5.4.0"
},
"input_type" : "log",
"message" : "Aug 01 10:40:02 Updated: libxml2-2.9.1-6.el7_2.3.x86_64",
"offset" : 640,
"source" : "/var/log/yum.log",
"type" : "log"
}
},
{
"_index" : "filebeat-2017.05.25",
"_type" : "log",
"_id" : "AVxBPPh4yM45-hZBb-Fv",
"_score" : 1.0,
"_source" : {
"@timestamp" : "2017-05-25T20:12:08.669Z",
"beat" : {
"hostname" : "elk.example.com",
"name" : "elk.example.com",
"version" : "5.4.0"
},
"input_type" : "log",
"message" : "[\u001B[32m OK \u001B[0m] Reached target Remote File Systems.",
"offset" : 512,
"source" : "/var/log/boot.log",
"type" : "log"
}
},
{
"_index" : "filebeat-2017.05.25",
"_type" : "log",
"_id" : "AVxBPPh4yM45-hZBb-F0",
"_score" : 1.0,
"_source" : {
"@timestamp" : "2017-05-25T20:12:08.669Z",
"beat" : {
"hostname" : "elk.example.com",
"name" : "elk.example.com",
"version" : "5.4.0"
},
"input_type" : "log",
"message" : "Aug 01 10:40:14 Updated: pcre-8.32-15.el7_2.1.x86_64",
"offset" : 814,
"source" : "/var/log/yum.log",
"type" : "log"
}
},
}
]
}
}

In Kibana:

Table:

JSON:

{
"_index": "filebeat-2017.06.13",
"_type": "syslog",
"_id": "AVyiSt1v_ojTDgCejFaf",
"_version": 1,
"_score": null,
"_source": {
"syslog_pid": "1407",
"syslog_severity_code": 5,
"offset": 78,
"syslog_facility": "user-level",
"count": 1,
"input_type": "log",
"syslog_facility_code": 1,
"source": "/var/log/messages",
"syslog_program": "bash",
"message": "Jun 13 11:29:45 elkclient bash[1407]: root [ ] : #011 echo "" > /var/log/messages",
"type": "syslog",
"syslog_message": "root [ ] : #011 echo "" > /var/log/messages",
"syslog_severity": "notice",
"tags": [
"beats_input_codec_plain_applied"
],
"received_from": "elkclient.example.com",
"@timestamp": "2017-06-13T16:29:45.000Z",
"syslog_hostname": "elkclient",
"syslog_timestamp": "Jun 13 11:29:45",
"received_at": "2017-06-13T12:30:30.614Z",
"@version": "1",
"beat": {
"hostname": "elkclient.example.com",
"name": "elkclient.example.com"
},
"host": "elkclient.example.com",
"fields": null
},
"fields": {
"@timestamp": [
1497371385000
]
},
"sort": [
1497371385000
]
}

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.