Syslog_severity_code it's always "5" , syslog_facility it's always "1", syslog_facility it's always "user-level" and syslog_severity it's always "notice"

Julian_Tete · June 9, 2017, 8:38pm

Hello ELK's Friends

I am trying to set up an ELK system to replace an Rsyslog + Adiscon LogAnalyzer implementation. The operating system used was CentOS 7.3 The versions of the installed packages were the following:

filebeat.x86_64 0:5.4.1-1

kibana.x86_64 0:5.4.1-1

logstash.noarch 1:5.4.1-1

elasticsearch.noarch 0:5.4.1-1

The Logstash configuration files are as follows:

/etc/logstash/conf.d/input.conf

input {
beats {
port => 5044
ssl => true
ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
}
}

/etc/logstash/conf.d/output.conf

output {
elasticsearch {
hosts => ["localhost:9200"]
sniffing => true
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}

/etc/logstash/conf.d/filter.conf

filter {
if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:[%{POSINT:syslog_pid}])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
}
syslog_pri { }
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}

In the client operating system (CentOS 7.3)

FileBeat was configured as follows:

(No comment)

/etc/filebeat/filebeat.yml

filebeat:
prospectors:
-
paths:
- /var/log/*.log
- /var/log/secure
- /var/log/dmesg
- /var/log/messages
input_type: log
document_type: syslog
registry_file: /var/lib/filebeat/registry
output:
elasticsearch:
hosts: ["localhost:9200"]
logstash:
hosts: ["elk.example.com:5044"]
tls:
certificate_authorities: ["/etc/pki/tls/certs/logstash-forwarder.crt"]
shipper:
logging:
files:

But in Kibana, all events are classified as well

syslog_severity_code 5 or -

syslog_facility user-level or -

Any help will be greatly appreciated.

Makra · June 10, 2017, 1:27am

Hi
Can you make paths: section entry one line like

paths:
- /var/log/messages

Also comment/remove the following logstash hosts section.

logstash:
hosts: ["elk.example.com:5044"]

Restart the logstash and run the following in the terminal -

$ tail -f /var/log/messages

Now check whether all the logs that have been shown by above command are transferred to kibana dashboard.

Regards

Julian_Tete · June 12, 2017, 4:24pm

Hi Makra

But if I remove the hosts section, How can I found the S.O Server, from the S.O Client ?

Makra · June 13, 2017, 12:20am

Hi
Julian
I assume your logstash host is remote one where you are sending the logs directly (port 5044) and ES is localhost. Now, Do you want the logs to be transferred to ES and logstash host simultaneously ?
It looks to me your configuration is correct. What is the result of tail command ? Does the same logs appear in the Kibana ? If that is the case then you need check the syslog configuration.

magnusbaeck · June 13, 2017, 8:36am

The syslog_pri filter reads a field with the encoded facility and severity information. By default it reads the syslog_pri field. Do you have a syslog_pri field?

Julian_Tete · June 13, 2017, 4:04pm

Yeah that's the idea Makra simultaneously. Tail a which archive Makra ?

Julian_Tete · June 13, 2017, 4:05pm

Hi magnusbaeck

Please look at this:

/etc/logstash/conf.d/filter.conf

filter {
if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:[%{POSINT:syslog_pid}])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
}
syslog_pri { }
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}

Makra · June 13, 2017, 4:18pm

Add log files one by one. Lets tail syslog first and check that the logs do arrive in the kibana. If you have something wrong in logstash filter configuration then there will be some events that you can see in the output of tail command but will not be there in kibana. This will make sure that the logstash configuration is correct.

Make filebeat paths: section points to /var/log/syslog

paths:
- /var/log/messages
.................................................

Check the syslog that arrives in the files.

$ tail -f /var/log/syslog

Hope this helps

Julian_Tete · June 13, 2017, 4:46pm

Mmmm my setup is like this elkserver.example.com (That server hosts Elesticsearch, Logstash and Kibana) and elkclient (Only has filebeat)

In elkclient.example.com with only /var/log/messages path

tail -f /var/log/messages

Jun 13 11:31:25 elkclient systemd: Time has been changed
Jun 13 11:31:25 elkclient chronyd[813]: System clock was stepped by 14401.230573 seconds
Jun 13 11:31:32 elkclient systemd: Created slice user-1000.slice.
Jun 13 11:31:32 elkclient systemd: Starting user-1000.slice.
Jun 13 11:31:32 elkclient systemd: Started Session 2 of user cnscadmin.
Jun 13 11:31:32 elkclient systemd-logind: New session 2 of user cnscadmin.
Jun 13 11:31:32 elkclient systemd: Starting Session 2 of user cnscadmin.
Jun 13 11:31:33 elkclient bash[1300]: cnscadmin [ 192.168.7.126 ] : #011 su -
Jun 13 11:31:37 elkclient su: (to root) cnscadmin on pts/0
Jun 13 11:31:38 elkclient bash[1359]: root [ ] : #011 systemctl reboot

In elkserver.example.com

curl -XGET 'http://localhost:9200/filebeat-*/_search?pretty':

{
"took" : 3,
"timed_out" : false,
"_shards" : {
"total" : 50,
"successful" : 50,
"failed" : 0
},
"hits" : {
"total" : 42218,
"max_score" : 1.0,
"hits" : [
{
"_index" : "filebeat-2017.05.25",
"_type" : "log",
"_id" : "AVxBPPh4yM45-hZBb-Ff",
"_score" : 1.0,
"_source" : {
"@timestamp" : "2017-05-25T20:12:08.669Z",
"beat" : {
"hostname" : "elk.example.com",
"name" : "elk.example.com",
"version" : "5.4.0"
},
"input_type" : "log",
"message" : "Aug 01 10:40:01 Updated: systemd-libs-219-19.el7_2.11.x86_64",
"offset" : 300,
"source" : "/var/log/yum.log",
"type" : "log"
}
},
{
"_index" : "filebeat-2017.05.25",
"_type" : "log",
"_id" : "AVxBPPh4yM45-hZBb-Fh",
"_score" : 1.0,
"_source" : {
"@timestamp" : "2017-05-25T20:12:08.669Z",
"beat" : {
"hostname" : "elk.example.com",
"name" : "elk.example.com",
"version" : "5.4.0"
},
"input_type" : "log",
"message" : "Aug 01 10:40:01 Updated: nss-util-3.21.0-2.2.el7_2.x86_64",
"offset" : 410,
"source" : "/var/log/yum.log",
"type" : "log"
}
},
{
"_index" : "filebeat-2017.05.25",
"_type" : "log",
"_id" : "AVxBPPh4yM45-hZBb-Fj",
"_score" : 1.0,
"_source" : {
"@timestamp" : "2017-05-25T20:12:08.669Z",
"beat" : {
"hostname" : "elk.example.com",
"name" : "elk.example.com",
"version" : "5.4.0"
},
"input_type" : "log",
"message" : "Aug 01 10:40:01 Updated: libgudev1-219-19.el7_2.11.x86_64",
"offset" : 526,
"source" : "/var/log/yum.log",
"type" : "log"
}
},
{
"_index" : "filebeat-2017.05.25",
"_type" : "log",
"_id" : "AVxBPPh4yM45-hZBb-Fk",
"_score" : 1.0,
"_source" : {
"@timestamp" : "2017-05-25T20:12:08.669Z",
"beat" : {
"hostname" : "elk.example.com",
"name" : "elk.example.com",
"version" : "5.4.0"
},
"input_type" : "log",
"message" : "[\u001B[32m OK \u001B[0m] Started Show Plymouth Boot Screen.",
"offset" : 54,
"source" : "/var/log/boot.log",
"type" : "log"
}
},
{
"_index" : "filebeat-2017.05.25",
"_type" : "log",
"_id" : "AVxBPPh4yM45-hZBb-Fm",
"_score" : 1.0,
"_source" : {
"@timestamp" : "2017-05-25T20:12:08.669Z",
"beat" : {
"hostname" : "elk.example.com",
"name" : "elk.example.com",
"version" : "5.4.0"
},
"input_type" : "log",
"message" : "Aug 01 10:40:02 Updated: libxml2-2.9.1-6.el7_2.3.x86_64",
"offset" : 640,
"source" : "/var/log/yum.log",
"type" : "log"
}
},
{
"_index" : "filebeat-2017.05.25",
"_type" : "log",
"_id" : "AVxBPPh4yM45-hZBb-Fv",
"_score" : 1.0,
"_source" : {
"@timestamp" : "2017-05-25T20:12:08.669Z",
"beat" : {
"hostname" : "elk.example.com",
"name" : "elk.example.com",
"version" : "5.4.0"
},
"input_type" : "log",
"message" : "[\u001B[32m OK \u001B[0m] Reached target Remote File Systems.",
"offset" : 512,
"source" : "/var/log/boot.log",
"type" : "log"
}
},
{
"_index" : "filebeat-2017.05.25",
"_type" : "log",
"_id" : "AVxBPPh4yM45-hZBb-F0",
"_score" : 1.0,
"_source" : {
"@timestamp" : "2017-05-25T20:12:08.669Z",
"beat" : {
"hostname" : "elk.example.com",
"name" : "elk.example.com",
"version" : "5.4.0"
},
"input_type" : "log",
"message" : "Aug 01 10:40:14 Updated: pcre-8.32-15.el7_2.1.x86_64",
"offset" : 814,
"source" : "/var/log/yum.log",
"type" : "log"
}
},
}
]
}
}

In Kibana:

Table:

JSON:

{
"_index": "filebeat-2017.06.13",
"_type": "syslog",
"_id": "AVyiSt1v_ojTDgCejFaf",
"_version": 1,
"_score": null,
"_source": {
"syslog_pid": "1407",
"syslog_severity_code": 5,
"offset": 78,
"syslog_facility": "user-level",
"count": 1,
"input_type": "log",
"syslog_facility_code": 1,
"source": "/var/log/messages",
"syslog_program": "bash",
"message": "Jun 13 11:29:45 elkclient bash[1407]: root [ ] : #011 echo "" > /var/log/messages",
"type": "syslog",
"syslog_message": "root [ ] : #011 echo "" > /var/log/messages",
"syslog_severity": "notice",
"tags": [
"beats_input_codec_plain_applied"
],
"received_from": "elkclient.example.com",
"@timestamp": "2017-06-13T16:29:45.000Z",
"syslog_hostname": "elkclient",
"syslog_timestamp": "Jun 13 11:29:45",
"received_at": "2017-06-13T12:30:30.614Z",
"@version": "1",
"beat": {
"hostname": "elkclient.example.com",
"name": "elkclient.example.com"
},
"host": "elkclient.example.com",
"fields": null
},
"fields": {
"@timestamp": [
1497371385000
]
},
"sort": [
1497371385000
]
}

system · July 11, 2017, 4:46pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
All syslogs have same severity Logstash	4	478	September 20, 2019
Centos 7.5 with rsyslog 8.24 / logstash 6.3.0 - on logstash output always severity code 5 and facility code 1 - Other post found on but still not solved Logstash	2	1718	August 14, 2018
Logstash, syslog severity, etc Logstash	2	2069	July 6, 2017
Configuring logstash-input-syslog Logstash	5	9477	June 1, 2018
Syslog severity and facility not set when upgrading version Logstash	4	308	November 27, 2023

Syslog_severity_code it's always "5" , syslog_facility it's always "1", syslog_facility it's always "user-level" and syslog_severity it's always "notice"

Related topics