Kibana takes time to show data in Discover

Elastic Stack Kibana
1

Hello,

Elasticsearch is getting data (way enough to show on Kibana). But my problem is that Kibana starts showing data only a few hours after its start. Meaning I miss the visualization of hours of data.

Any idea?

2

that shouldn't be the case ... can you see your data in discover mode ? (select time range last 30 minutes and see if any data is there for your selected index pattern)

3

That's the problem. Discover doesn't show any data. It starts showing around 5hours after I start ELK.

4

Elasticsearch and Kibana requires all timestamp fields to be in UTC timezone. Can you verify that timestamps in indexed data is indeed in UTC?

1 Like
5

I don't know if the timestamp is UTC but why would Kibana starts showing logs after a few hours? If the timestamp would be wrong Kibana wouldn't show anything I guess.

How should I know id the data is in UTC?

6

Look at the log entries in discover mode, and inspect the JSON. What do the timestamp(s) look like? How are you processing timestamps in your ingest pipeline?

Based on your description another possible cause could be that your documents are being delayed throughout the ingest process. What does your pipeline look like?

1 Like
7

Timestamp :

"@timestamp": [
      1497862009000
    ]

Pipeline

filter {
    grok {
        add_tag => [ "firewall" ]
        match => [ "message", "<(?<evtid>.*)>(?<datetime>(?:Jan(?:uary)?|Feb(?:ruary)?|Mar(?:ch)?|Apr(?:il)?|May|Jun(?:e)?|Jul(?:y)?|Aug(?:ust)?|Sep(?:tember)?|Oct(?:ober)?|Nov(?:ember)?|Dec(?:ember)?)\s+(?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9]) (?:2[0123]|[01]?[0-9]):(?:[0-5][0-9]):(?:[0-5][0-9])) (?<prog>.*?): (?<msg>.*)" ]
    }
    mutate {
        gsub => ["datetime", "  ", " "]
    }
    date {
        match => [ "datetime", "MMM dd HH:mm:ss" ]
    }
    mutate {
        replace => [ "message", "%{msg}" ]
    }
    mutate {
        remove_field => [ "msg", "datetime" ]
    }
    if [prog] =~ /^filterlog$/ {
        mutate {
            remove_field => [ "msg", "datetime" ]
        }
        grok {
            patterns_dir => "/etc/logstash/conf.d/patterns"
            match => [ "message", "%{PFSENSE_LOG_DATA}%{PFSENSE_IP_SPECIFIC_DATA}%{PFSENSE_IP_DATA}%{PFSENSE_PROTOCOL_DATA}",
           "message", "%{PFSENSE_LOG_DATA}%{PFSENSE_IPv4_SPECIFIC_DATA_ECN}%{PFSENSE_IP_DATA}%{PFSENSE_PROTOCOL_DATA}" ]
        }
        mutate {
            lowercase => [ 'proto' ]
        }
        geoip {
            add_tag => [ "GeoIP" ]
            source => "src_ip"
            # Optional GeoIP database
            # Comment out the below if you do not wise to utilize and omit last three steps dealing with (recommended) suffix
            # database => "/usr/share/GeoIP/GeoLiteCountry.dat"
        }
    }
}

The output sends everything to Elasticsearch (that part works perfectly).

8

Can you share a sample event that has shown up late? Which time zone are you in?

9

My timezone is UTC + 2 (summer time).

No data showed up late. Kibana simply starts showing data after a few hours, meaning that the first hours the data gets to Elasticsearch but NEVER show up in Kibana.

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.