# Incorrect Timestamp in Machine Learning Job Result

**URL:** <https://discuss.elastic.co/t/incorrect-timestamp-in-machine-learning-job-result/99111>\
**Category:** Elasticsearch\
**Tags:** elastic-stack-machine-learning\
**Created:** [September 1, 2017, 10:12am UTC](https://discuss.elastic.co/t/incorrect-timestamp-in-machine-learning-job-result/99111 "2017-09-01T10:12:44Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![nktvys](https://avatars.discourse-cdn.com/v4/letter/n/a183cd/32.png) [@nktvys](https://discuss.elastic.co/u/nktvys)\
**Post date:** [September 1, 2017, 10:12am UTC](https://discuss.elastic.co/t/incorrect-timestamp-in-machine-learning-job-result/99111/1 "2017-09-01T10:12:44Z")

</div>

Hi,

I have a few documents that have a timestamp of 01:46:00:038. But, after running a ML job, the results show these documents with a different time 07:16:00:038. I guess this is due to time difference of 5:30 but I do not know why this is happening. Can you please provide a solution to fix this issue?

Thanks!

---

<div class="post-metadata">

**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\
**Post date:** [September 1, 2017, 2:58pm UTC](https://discuss.elastic.co/t/incorrect-timestamp-in-machine-learning-job-result/99111/2 "2017-09-01T14:58:22Z")

</div>

Kibana will show events in your local timezone unless you change that option:

[https://www.elastic.co/guide/en/kibana/current/advanced-options.html](https://www.elastic.co/guide/en/kibana/current/advanced-options.html)

---

<div class="post-metadata">

**Author:** ![nktvys](https://avatars.discourse-cdn.com/v4/letter/n/a183cd/32.png) [@nktvys](https://discuss.elastic.co/u/nktvys)\
**Post date:** [September 6, 2017, 6:40am UTC](https://discuss.elastic.co/t/incorrect-timestamp-in-machine-learning-job-result/99111/3 "2017-09-06T06:40:19Z")

</div>

Hi,

I changed the time zone to GMT. Now, @timestamp matches the timestamp in the log file. However, in the machine learning module, the timestamp is still showing in GMT+5:30 (my local time).

Why isnt the machine learning job taking the value in @timestamp?

---

<div class="post-metadata">

**Author:** ![Peter\_Harverson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/peter_harverson/32/21057_2.png) [@Peter\_Harverson](https://discuss.elastic.co/u/Peter_Harverson)\
**Post date:** [September 6, 2017, 2:57pm UTC](https://discuss.elastic.co/t/incorrect-timestamp-in-machine-learning-job-result/99111/4 "2017-09-06T14:57:23Z")

</div>

Hi Niket,

Just to clarify - did you change the time zone to GMT for the 'dateFormat:tz' under the Advanced Settings tab of the Kibana Management page?

Unfortunately the visuals in the ML plugin will only currently display times in the local timezone of your browser (with dateFormat:tz set to the default of 'Browser'). This is something we will look to address in a subsequent release.

Thanks  
Pete

---

<div class="post-metadata">

**Author:** ![nktvys](https://avatars.discourse-cdn.com/v4/letter/n/a183cd/32.png) [@nktvys](https://discuss.elastic.co/u/nktvys)\
**Post date:** [September 7, 2017, 5:02am UTC](https://discuss.elastic.co/t/incorrect-timestamp-in-machine-learning-job-result/99111/5 "2017-09-07T05:02:25Z")

</div>

Hi Peter,

Yes, I did change the time zone to GMT for the ‘dateFormat:tz’. When I want to send alerts for anomalies detected, the timestamp for those anomalies is incorrect due to the ML result. Can you help me solve this? If not then I'll wait for the next release.

Thanks!

---

<div class="post-metadata">

**Author:** ![droberts195](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/droberts195/32/17692_2.png) [@droberts195](https://discuss.elastic.co/u/droberts195)\
**Post date:** [September 7, 2017, 9:05am UTC](https://discuss.elastic.co/t/incorrect-timestamp-in-machine-learning-job-result/99111/6 "2017-09-07T09:05:55Z")

</div>

Is it possible that the data you are analyzing contains timestamps in text format in the India timezone but without the timezone in the textual representation? In this case you'll end up with a `_source` document that shows your original textual representation, which you'll interpret as India time. But Elasticsearch will have converted it to epoch milliseconds to store in the doc values on the assumption it's UTC. Then ML will analyze it and create results based on the epoch milliseconds stored in the doc values.

This thread discusses the way Elasticsearch converts textual timestamps assuming UTC unless the text contains a timezone: [Storing Dates in Elastic Best Practices](https://discuss.elastic.co/t/storing-dates-in-elastic-best-practices/57496/2)

If what I'm suggesting isn't clear then it might be easier to discuss a specific example. If you paste the `_source` of one of the documents you're analyzing then I can talk specifically about it.

---

<div class="post-metadata">

**Author:** ![nktvys](https://avatars.discourse-cdn.com/v4/letter/n/a183cd/32.png) [@nktvys](https://discuss.elastic.co/u/nktvys)\
**Post date:** [September 7, 2017, 9:27am UTC](https://discuss.elastic.co/t/incorrect-timestamp-in-machine-learning-job-result/99111/7 "2017-09-07T09:27:07Z")

</div>

Here is a sample `_source` document:

```
"_source": {
          "job_id": "test_job",
          "result_type": "record",
          "probability": 0.00684734,
          "record_score": 35.3922,
          "initial_record_score": 35.3922,
          "bucket_span": 300,
          "detector_index": 1,
          "is_interim": false,
          "timestamp": 1503741600000,
          "by_field_name": "mlcategory",
          "by_field_value": "19",
          "function": "non_zero_count",
          "function_description": "count",
          "typical": [
            1.03523
          ],
          "actual": [
            6
          ],
          "influencers": [
            {
              "influencer_field_name": "sampler.keyword",
              "influencer_field_values": [
                "101"
              ]
            }
          ],
          "sampler.keyword": [
            "101"
          ],
          "mlcategory": [
            "19"
          ]
        }
```

---

<div class="post-metadata">

**Author:** ![droberts195](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/droberts195/32/17692_2.png) [@droberts195](https://discuss.elastic.co/u/droberts195)\
**Post date:** [September 7, 2017, 9:40am UTC](https://discuss.elastic.co/t/incorrect-timestamp-in-machine-learning-job-result/99111/8 "2017-09-07T09:40:13Z")

</div>

Ah, OK. So your timestamps are already in milliseconds since the epoch in the source documents. So timezone conversions on indexing cannot be the problem. My theory was wrong then.

---

<div class="post-metadata">

**Author:** ![Peter\_Harverson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/peter_harverson/32/21057_2.png) [@Peter\_Harverson](https://discuss.elastic.co/u/Peter_Harverson)\
**Post date:** [September 7, 2017, 10:00am UTC](https://discuss.elastic.co/t/incorrect-timestamp-in-machine-learning-job-result/99111/9 "2017-09-07T10:00:42Z")

</div>

OK, so seeing as timezone conversions on indexing aren't the root of the problem, hopefully your issue will be addressed when we fix the ML pages to observe the Kibana ‘dateFormat:tz’ configuration option when set to something other than 'Browser'. Till then the times in the ML pages will be offset to GMT according to the local timezone of your browser. I'll provide an update when that is fixed.

---

<div class="post-metadata">

**Author:** ![nktvys](https://avatars.discourse-cdn.com/v4/letter/n/a183cd/32.png) [@nktvys](https://discuss.elastic.co/u/nktvys)\
**Post date:** [September 7, 2017, 10:16am UTC](https://discuss.elastic.co/t/incorrect-timestamp-in-machine-learning-job-result/99111/10 "2017-09-07T10:16:35Z")

</div>

OK. I'll write a small note regarding the timestamp issue while alerting for now. I hope this issue gets resolved soon.

Thanks Peter!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 5, 2017, 10:17am UTC](https://discuss.elastic.co/t/incorrect-timestamp-in-machine-learning-job-result/99111/11 "2017-10-05T10:17:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
