Where does Kibana get the timepicker's "now" value?

[hansell.baran (Hansell)]

Hello!

I noticed that when I use Kibana's "Quick" timepickers, the most recent data shown is 2 hours behind my browser's time. This only happens with the "Last..." options. If I use "Today", all the data is shown.

It is worth mentioning that the server (Ubuntu 18.04) where Kibana runs has UTC (running date command returns "Wed Apr 10 16:17:15 UTC 2019"). I tried changing the timezone but it does not help.

Where does Kibana get the "now" date?

Some additional details:

I configured Kibana to use UTC instead of the default Browser setting (Kibana -> Managament -> Advanced Settings -> dateFormat:tz) (If I do not use UTC, then log timestamps do not match Kibana timestamps)

From this post,

6.2.0 - Some "Quick time ranges" have been removed

Hi there, I upgraded my dev environnement in version 6.2.0 this morning and observed that some time ranges I used to select in "Quick" section are no longer available. Including "Yesterday" and "Day before yesterday" among many others. Is it intentional ? According to me, those time ranges are necessary for people running daily batches process. Kuaaaly

I got to know that there is a timepicker:quickRanges setting that I modified to "make it work" for the "Last 7 days" option. I changed this:
{
"from": "now-7d",
"to": "now",
"display": "Last 7 days",
"section": 1
}
into this:
{
"from": "now-7d",
"to": "now/d",
"display": "Last 7 days",
"section": 1
}

However, I would like ALL "Last..." options to use the now date that my browser is using which should happen by using "Browser" in the dateFormat:tz setting.

How should I fix this issue?

Here are some screen shots to visualize the problem:

Kibana with dateFormat:tz = UTC and default "Quick" timepicker options:

Kibana with dateFormat:tz = Browser and default "Quick" timepicker options:

Kibana with dateFormat:tz = UTC and modified "Quick -> Last 7 days" timepicker option (notice that more data is visualized as the "to" property is set to "now/d"):

Any help would be greatly appreciated! Thanks!

[Marius_Dragomir]

Can you check the request in the inspector (top right) for all these options and see if what the actual UNIX timestamps are used for the request and filters?

[hansell.baran (Hansell)]

Thanks for your response @Marius_Dragomir...

I believe the "now" is from perhaps calling Javascript "Date.now()" or something similar (I did not look into the source code). However, I solved the issue by modifying the Elasticsearch pipelines.

Since there is no detailed documentation regarding the Filebeat modules and how their pipelines are structured and how they work, i.e., how they parse dates (timezones specifically), and I am not an ELK expert, I assumed that the parsed timestamp "would be accurrate" (consider the timezone if there is one), however, since all the logs in my server have timezone GMT+2, I didn't realise that the Filebeat Nginx module was parsing the logs considering they were GMT+0.

I modified ALL the Elasticsearch pipelines (specifically those related to the filebeat / nginx modules), so that the "date" processor would have an extra property "timezone:" "CEST", reverted Kibana to "Browser" and now logs are showing up with the correct timestamp and the "Quick" "Last x..." options work as expected! Awesome!

[Marius_Dragomir]

Ah yes, Elasticsearch by default will assume UTC unless you specify a timezone.