Can't use anymore @timestamp after a time filter change

Elastic Stack Kibana
1

Hello,

Usually, I always used @timestamp as time filter field name for Kibana index pattern. And I never used a date filter plugin in logstash.

Yesterday, for testing purpose, I set a date filter in logstash :

filter {
date {
    match => [ "timestamp","UNIX" ]
    target => "log_timestamp"
    timezone => "Europe/Paris"
}
}

Then I made a new index pattern in Kibana. Everything was good.

But when I came back to old configuration by removing date filter section in logstash, and with a new index pattern in Kibana using @timestamp as time filter field name, no more results in Kibana. Index are found, documents number increases but Kibana show nothing....
Last logs dislayed date from the change. Like if the change has broke something...

Here is a fresh log sample (ruby debug logstash) :

{
         "client_ip" => "1.2.3.4",
                "to" => "recipient_name@domain.com",
                "vd" => "root",
           "message" => "timestamp=1574845845 date=2019-11-27 time=10:10:45.818 devname=\"DEVICE1\" device_id=\"DEVICE1\" log_id=\"0300011170\" type=\"spam\" pri=\"information\" session_id=\"xAR9AjuX011169-xAR9AjuZ011169\" client_name=\"yolo.net\" client_ip=\"1.2.3.4\" dst_ip=\"10.255.50.205\" from=\"jeandenis@palombiere.com\" to=\"recipient_name@domain.com\" subject=\"Villegoudou\"  nested=\"File name: image001.jpg scanned by Antivirus Scannerclean Attachment Filterclean\" vd=\"root\"\n",
          "facility" => 23,
    "facility_label" => "local7",
       "identifiant" => "customer1",
              "time" => "10:10:45.818",
              "type" => "spam",
       "client_name" => "yolo.net",
             "geoip" => {
          "country_name" => "France",
        "continent_code" => "EU",
              "location" => {
            "lon" => 2.3387000000000002,
            "lat" => 48.8582
        },
         "country_code2" => "FR",
             "longitude" => 2.3387000000000002,
         "country_code3" => "FR",
              "timezone" => "Europe/Paris",
                    "ip" => "1.2.3.4",
              "latitude" => 48.8582
    },
              "from" => "jeandenis@palombiere.com",
           "subject" => "Villegoudou",
        "session_id" => "xAR9AjuX011169-xAR9AjuZ011169",
         "timestamp" => "1574845845",
              "host" => "192.168.160.5",
    "severity_label" => "Informational",
               "pri" => "information",
     "log_timestamp" => 2019-11-27T09:10:45.000Z,
          "priority" => 190,
         "logsource" => "31.217.232.132",
           "devname" => "DEVICE1",
            "nested" => "File name: image001.jpg scanned by Antivirus Scannerclean Attachment Filterclean",
          "@version" => "1",
         "device_id" => "DEVICE1",
        "@timestamp" => 2019-11-27T08:10:46.000Z,
            "dst_ip" => "10.255.50.205",
            "log_id" => "0300011170",
          "severity" => 6
}

Capture
Capture.PNG1163×305 24.5 KB

I don't understand why Kibana show nothing as I have a @timestamp field..Any idea to figure this out ?

Thanks ! :slight_smile:

2

OK. I remember, I've done some timezone change on my host. I think this has introduced some confusion in logstash conf and kibana display

Solved by going back to previous settings : timedatectl set-timezone Etc/UTC

Sorry for the noise :wink:

3

Glad it's was "automatically" resolved :smiley: have a nice day

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.