# Datafeed not happening in ml job **URL:** \ **Category:** Kibana\ **Tags:** elastic-stack-machine-learning\ **Created:** [December 19, 2018, 9:47am UTC](https://discuss.elastic.co/t/datafeed-not-happening-in-ml-job/161483 "2018-12-19T09:47:23Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Sripal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sripal/32/36704_2.png) [@Sripal](https://discuss.elastic.co/u/Sripal)\ **Post date:** [December 19, 2018, 9:47am UTC](https://discuss.elastic.co/t/datafeed-not-happening-in-ml-job/161483/1 "2018-12-19T09:47:24Z")
Hi all, I'm new to ml job. I've created the ml job in kibana. I got an error ![image](https://us1.discourse-cdn.com/elastic/original/3X/f/e/fe569176b88ade528c22fcef2a8e7994bd6660e1.png) **1. My data with timestamp (It's a sample data of one log file ) all data are feed at one shot.** ![image](https://us1.discourse-cdn.com/elastic/original/3X/9/a/9a9d37bf45defc74666f8d970b58c400f6be6d25.png) **2. While validating my job i got the below warnings.** ![image](https://us1.discourse-cdn.com/elastic/original/3X/f/3/f31d3b0662c6769c8999a647975a144211a1f6c2.png) **3. In job management tab i got only one proceed data for my job.** ![image](https://us1.discourse-cdn.com/elastic/original/3X/0/4/04801169420cedbd4e60f62eda38d0d90133447d.png) **4. My job config** ![image](https://us1.discourse-cdn.com/elastic/original/3X/7/8/7807631842ba55f1fa6490c0e639682d886c255e.png) **5. JSON file** ![image](https://us1.discourse-cdn.com/elastic/original/3X/c/4/c425a706ea0cd4381d12b694ab9908665639bf88.png) ![image](https://us1.discourse-cdn.com/elastic/original/3X/0/d/0d61cbcfba2c5632301e3cac23f02b9a7d291435.png) ![image](https://us1.discourse-cdn.com/elastic/original/3X/d/4/d4335f46a55048dddbd0cb2eca648ce30832d26e.png) ![image](https://us1.discourse-cdn.com/elastic/original/3X/7/f/7f23d9496a85755ea20538ae3e8f7a8c4e5980a5.png) Can anybody help me. Appreciate your responses ---
**Author:** ![Sripal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sripal/32/36704_2.png) [@Sripal](https://discuss.elastic.co/u/Sripal)\ **Post date:** [December 19, 2018, 11:44am UTC](https://discuss.elastic.co/t/datafeed-not-happening-in-ml-job/161483/2 "2018-12-19T11:44:06Z")
I have 100 thousand documents in elastic search but it is processed only 1. please let me know my mistake. **My expectation in ml job is:** 1. filed "response" is the response time of the HTTP request. I want to find the mean of "response" and to predict the mean value in future. 2. I tried SUM also but, i got same response. Anyone guide me how to do ml job. ---
**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\ **Post date:** [December 19, 2018, 2:19pm UTC](https://discuss.elastic.co/t/datafeed-not-happening-in-ml-job/161483/3 "2018-12-19T14:19:00Z")
It looks like the main crux of your problem is that although you have 100k+ documents in Elasticsearch, they all have the same timestamp. In logstash, you must have not properly extracted the timestamp from the logs and used that to write the `@timestamp` field (??) See the "Date filter" section of: [https://www.elastic.co/blog/a-practical-introduction-to-logstash](https://www.elastic.co/blog/a-practical-introduction-to-logstash) The ML job needs to see documents over long-ish periods of time (hours, days, weeks, etc.). Until you fix the timestamps, your ML jobs will never work. ---
**Author:** ![Sripal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sripal/32/36704_2.png) [@Sripal](https://discuss.elastic.co/u/Sripal)\ **Post date:** [December 20, 2018, 9:13am UTC](https://discuss.elastic.co/t/datafeed-not-happening-in-ml-job/161483/4 "2018-12-20T09:13:59Z")
Hi @richcollier , Thank you for the response. Please review the below screenshot of my log line ![image](https://us1.discourse-cdn.com/elastic/original/3X/8/e/8ed9732eab7e8acb6c2663043012934530886ad3.png) Here ;; 1. I tried to read the sample log file. 2. @timestamp is taken as when the log lines are read and sent to elastic search( **Dec 19 2018 xxxx** ). but the actual log time of log in the file is ( **Dec 4 2018 xxx** ) 3. currently the field timestamp (which i defined) in text type. I can mutate it to date. 4. How to i get the actual log time ( **Dec 4 2018 xxx** ) in elasticsearch??? could you please guide me how to achieve this. ---
**Author:** ![Sripal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sripal/32/36704_2.png) [@Sripal](https://discuss.elastic.co/u/Sripal)\ **Post date:** [December 20, 2018, 9:19am UTC](https://discuss.elastic.co/t/datafeed-not-happening-in-ml-job/161483/5 "2018-12-20T09:19:24Z")
@richcollier , I'll try this suggestion and let you know the results. ---
**Author:** ![Sripal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sripal/32/36704_2.png) [@Sripal](https://discuss.elastic.co/u/Sripal)\ **Post date:** [December 20, 2018, 11:03am UTC](https://discuss.elastic.co/t/datafeed-not-happening-in-ml-job/161483/6 "2018-12-20T11:03:57Z")
@richcollier Thanks much. Issue is resolved. I mapped my logtime to the timestamp. filter { grok { match =\> { "message" =\> '%{NOTSPACE:clientip}:- %{NOTSPACE:user} %{NOTSPACE:pass} %{NOTSPACE:role} [%{HTTPDATE:logtime}] "%{WORD:method} %{DATA:request} HTTP/%{NUMBER:httpversion}" %{NUMBER:responsecode} %{NUMBER:response} %{NOTSPACE:id1} %{NOTSPACE:id2} %{NOTSPACE:id3} %{NOTSPACE:id4} %{NOTSPACE:id5}' } } date { match =\> ["logtime", "dd'/'MMM'/'yyyy:HH:mm:ss Z"] ``` } ``` mutate { convert =\> { "responsecode" =\> "integer" "response" =\> "integer" } } } **Now its working perfectly** ![image](https://us1.discourse-cdn.com/elastic/original/3X/d/d/dd8e6abe3aab73c53145064af985af54ab40fca7.png) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 17, 2019, 11:03am UTC](https://discuss.elastic.co/t/datafeed-not-happening-in-ml-job/161483/7 "2019-01-17T11:03:58Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.