Logstash timestamp/time/date problems

Hi all I need some help.

I set my first elk this morning . I used this tutorial. https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-elk-stack-on-centos-7

At the end I want my websphere logs to go to logstash but first syslog has to work fine.

Now logstash is receiving input but I have few problems.

First the known problem with changing logstash timestamp with real timestamp from log file. I googled a lot but cant figure this out.

This is my syslog filter

filter {
  if [type] == "syslog" {
    grok {
      match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
      add_field => [ "received_at", "%{@timestamp}" ]
      add_field => [ "received_from", "%{host}" ]
    }
    syslog_pri { }
    date {
      match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ] 
    }
  }
}

Log sample from client
Mar 23 11:12:00 hostname systemd: Starting Session 2 of user root
Locale the same on both

Second problem is that log order is messed up.
This is kibana view.
Mar 23 13:01:01 hostname systemd: Started Session 5 of user root.
Mar 23 13:01:01 hostname systemd: Starting Session 5 of user root.
Mar 23 12:15:12 hostname journal: root[/root] 56092 ipaddress 22: tail -f /var/log/messages
Mar 23 12:15:14 hostname journal: root[/root] 56092 ipaddress 22: locale
Mar 23 12:10:17 hostname journal: root[/root] 56092 ipaddress 22: tail -f /var/log/messages
Mar 23 12:10:23 hostname journal: root[/root] 56092 ipaddress 22: tail -f /var/log/secure

You can see wrong time order, Order in log file on client i as it should be.

Please show what an example event looks like. Copy/paste from Kibana's JSON tab. No screenshot.

{
"_index": "filebeat-2017.03.23",
"_type": "log",
"_id": "AVr7VWS9vp0eQ-kwixiE",
"_score": null,
"_source": {
"@timestamp": "2017-03-23T13:22:36.001Z",
"offset": 564659,
"@version": "1",
"input_type": "log",
"beat": {
"hostname": "myserver",
"name": "myserver",
"version": "5.2.2"
},
"host": "myserver",
"source": "/var/log/messages",
"message": "Mar 23 14:22:27 myserverjournal: root[/root] ipaddress 54723 ipaddress 22: timedatectl",
"type": "log",
"tags": [
"beats_input_codec_plain_applied"
]
},
"fields": {
"@timestamp": [
1490275356001
]
},
"sort": [
1490275356001
]
}

That event has the type "log" and not "syslog" so your filters are never applied.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.