Hi All,
I have a quick query regarding a data source that I'm extracting log data from and funneling into Elasticsearch via Logstash (Filebeat).
If the data source logs are already in UTC and being stored in Elasticsearch as UTC, would you expect Kibana to automaticlaly take the stored @timestamp UTC value and convert that to local time of the client accessing the Kibana interface? Is this a correct assertion?
The reason why I'm asking is because the data is stored as UTC (sourced as UTC) but Kibana is displaying the data as UTC instead of NZT.
The source data is parsed using a grok filter followed directly by a date filter as shown below;
date {
match => [ "EventTime", "YYYY-MM-dd HH:mm:ss" ]
If all my current data sources are NZT at the source, and then I run that through the date filter, I see that the data is stored as NZT and kibana displays it correctly.
In this situation, is it appropriate to specify the timezone option specifying Pacific\Auckland to convert it from UTC to NZT at time of ingest?
Hoping to clarify that this is the correct course of action. If however the logic is wrong and I should be changing all data sources (current sourced as NZT) to UTC prior to ingestion to align to how the current data source is doing things, that would also be an equally viable way of resolving the issue without having to define the timezone in date/time parser.
This more a sanity check to verify my own understanding based on my current knowledge base - comfortable making what ever changes are recommended to ensure appropriate normalisation across the data I'm ingesting.
Thanks,
Andrew