# ML Datafeed lookback retrieved no data

**URL:** <https://discuss.elastic.co/t/ml-datafeed-lookback-retrieved-no-data/260104>\
**Category:** Elasticsearch\
**Tags:** elastic-stack-machine-learning\
**Created:** [January 4, 2021, 4:07pm UTC](https://discuss.elastic.co/t/ml-datafeed-lookback-retrieved-no-data/260104 "2021-01-04T16:07:09Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![TheHunter1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thehunter1/32/80190_2.png) [@TheHunter1](https://discuss.elastic.co/u/TheHunter1)\
**Post date:** [January 4, 2021, 4:07pm UTC](https://discuss.elastic.co/t/ml-datafeed-lookback-retrieved-no-data/260104/1 "2021-01-04T16:07:09Z")

</div>

I am trying to run the integrated jobs for winlogbeat data, and most of them are having an error `Datafeed lookback retrieved no data`.

I already had the same problem using Elasticsearch version 8.0.0 from code source, So I decided to install the version 7.10.1, and even the beats are 7.10.1, and still have the same error.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/4/d/4d0b72bf510827616011dbd9ee737b2b291378b0.png)

So to test, I tried to clone one of the jobs, but just clicking on `clone` I get an error

```auto
[request body.earliestMs]: expected value of type [number] but got [null]

```

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/8/4/84aeede18490b50ccfb5ecccf7f073388f195a9b.png)

and when I continue till the validation, I am getting an error:

```auto
Job Validation Error

[request body.duration.start]: expected value of type [number] but got [null]

```

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/7/9/791dfd9272d93769b42921a35dd82384912685eb.png)

knowing that I am using a dedicated node for Machine learning by configuring the node as bellow:

```auto
node.roles: [ml]

```

Could you please help me to figure out why I am getting this error ?  
Thanks

---

<div class="post-metadata">

**Author:** ![droberts195](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/droberts195/32/17692_2.png) [@droberts195](https://discuss.elastic.co/u/droberts195)\
**Post date:** [January 5, 2021, 1:05pm UTC](https://discuss.elastic.co/t/ml-datafeed-lookback-retrieved-no-data/260104/2 "2021-01-05T13:05:32Z")

</div>

`[request body.duration.start]: expected value of type [number] but got [null]` is due to a bug in the UI that was fixed in [https://github.com/elastic/kibana/pull/86114](https://github.com/elastic/kibana/pull/86114). That fix will be in 7.11.

But both that and the original error of `Datafeed lookback retrieved no data` suggest that the datafeeds for your Winlogbeat jobs have ended up configured with searches that are not finding your Winlogbeat data. If you expand one of the rows on your jobs list you'll be able to see the datafeed config, including the indices that it's searching and the query it's using. Check whether you would expect a search using them to find your Winlogbeat data.

---

<div class="post-metadata">

**Author:** ![TheHunter1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thehunter1/32/80190_2.png) [@TheHunter1](https://discuss.elastic.co/u/TheHunter1)\
**Post date:** [January 5, 2021, 1:48pm UTC](https://discuss.elastic.co/t/ml-datafeed-lookback-retrieved-no-data/260104/3 "2021-01-05T13:48:39Z")

</div>

Thanks for your answer @droberts195,

Here is an example of the datafeed configuration of one of the jobs that are not working, and as it's shown in the picture bellow, it is searching data in the right indice `winlogbeat-*`,

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/c/a/caa3c2cc1082065e08833bfe4a7b0ba78f2af7db.png)

When I try:

```auto
GET _search
{
  "query": {
    "bool":{
      "filter":[{
        "term":{
          "event.action":"Process Create (rule: ProcessCreate)"
        }
      },
      {
        "term":{
          "agent.type":"winlogbeat"
        }
      }
      ]}
  }
}

```

I am getting no result too:

```auto
#! Deprecation: this request accesses system indices: [.apm-agent-configuration, .apm-custom-link, .async-search, .kibana_1, .kibana_task_manager_1, .ml-config, .security-7, .tasks, .transform-internal-005, .triggered_watches, .watches], but in a future major version, direct access to system indices will be prevented by default
{
  "took" : 2215,
  "timed_out" : false,
  "_shards" : {
    "total" : 36,
    "successful" : 36,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 0,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : []
  }
}

```

Is that mean that I don't have enough data ?

---

<div class="post-metadata">

**Author:** ![droberts195](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/droberts195/32/17692_2.png) [@droberts195](https://discuss.elastic.co/u/droberts195)\
**Post date:** [January 5, 2021, 2:33pm UTC](https://discuss.elastic.co/t/ml-datafeed-lookback-retrieved-no-data/260104/4 "2021-01-05T14:33:05Z")

</div>

> [@TheHunter1](#):
>
> Is that mean that I don't have enough data ?

Yes, try broadening the search to find out which of the two terms you don't have:

```auto
GET winlogbeat-*/_search
{
  "query": {
    "bool":{
      "filter":[{
        "term":{
          "event.action":"Process Create (rule: ProcessCreate)"
        }
      }
      ]}
  }
}

```

and:

```auto
GET winlogbeat-*/_search
{
  "query": {
    "bool":{
      "filter":[{
        "term":{
          "agent.type":"winlogbeat"
        }
      }
      ]}
  }
}

```

Do either of those find anything?

---

<div class="post-metadata">

**Author:** ![TheHunter1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thehunter1/32/80190_2.png) [@TheHunter1](https://discuss.elastic.co/u/TheHunter1)\
**Post date:** [January 5, 2021, 2:49pm UTC](https://discuss.elastic.co/t/ml-datafeed-lookback-retrieved-no-data/260104/5 "2021-01-05T14:49:20Z")

</div>

When I run the first query I am getting no result :

```auto
{
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 0,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : []
  }
}

```

And when I run the second query I get results.

I will let the winlogbeat to turn more in machins and then try again

Thanks for your help @droberts195 🙂

---

<div class="post-metadata">

**Author:** ![droberts195](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/droberts195/32/17692_2.png) [@droberts195](https://discuss.elastic.co/u/droberts195)\
**Post date:** [January 5, 2021, 3:14pm UTC](https://discuss.elastic.co/t/ml-datafeed-lookback-retrieved-no-data/260104/6 "2021-01-05T15:14:33Z")

</div>

> [@TheHunter1](#):
>
> When I run the first query I am getting no result

OK it might be worth having a browse through your `winlogbeat-*` indices using the Discover app. Do you see events that contain something similar to, but not identical to `Process Create (rule: ProcessCreate)`? I am wondering whether that string varies a bit between different versions of Windows, or whether it gets localized. For example, do you see documents in `winlogbeat-*` that have `Process Create` translated into some other language?

You might have discovered a flaw where these jobs have been accidentally tailored to one specific version of Windows or to Windows running in the US English locale only or something like that.

---

<div class="post-metadata">

**Author:** ![TheHunter1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thehunter1/32/80190_2.png) [@TheHunter1](https://discuss.elastic.co/u/TheHunter1)\
**Post date:** [January 5, 2021, 3:58pm UTC](https://discuss.elastic.co/t/ml-datafeed-lookback-retrieved-no-data/260104/7 "2021-01-05T15:58:06Z")

</div>

I used discover app to browse into my `winlogbeat-*` indice and then Toggle the column `event.Action` to see what I am getting inside.  
there was some result which are translated to French language, and some are in English, but I didn't find in both languages `Process Create (rule: ProcessCreate)`

here is a list of all what I get in `event.Action` field:

```auto
Événements d’arrêts inattendus d’applications
Ouvrir la session
group-membership-enumerated
Modification de l’état de la sécurité
Défragmentation en ligne
Général
Cliché instantané
Enregistrement/récupération
Modification de la stratégie d’audit
kerberos-service-ticket-renewed
kerberos-authentication-ticket-requested
Directory Service Access
user-member-enumerated
kerberos-service-ticket-requested
User Account Management
gent de mise à jour automatique Windows Update
logon-failed
General
Guest Library API
logged-out
credential-validated
logged-in-special
logged-in-explicit
logged-in

```

---

<div class="post-metadata">

**Author:** ![droberts195](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/droberts195/32/17692_2.png) [@droberts195](https://discuss.elastic.co/u/droberts195)\
**Post date:** [January 5, 2021, 4:25pm UTC](https://discuss.elastic.co/t/ml-datafeed-lookback-retrieved-no-data/260104/8 "2021-01-05T16:25:15Z")

</div>

A bit of searching led me to [https://github.com/elastic/kibana/issues/50003](https://github.com/elastic/kibana/issues/50003):

> Therein lies the problem. It is not obvious at first, but jobs like "windows\_anomalous\_process\_all\_hosts\_ecs" actually depend on the field event.action being a sysmon value "ProcessCreate" as opposed to the expected "created-process" that windows event id's give.

So it looks like what you are seeing is a problem that has been reported before but nothing has been done about it yet.

It seems that you have to enable Sysmon for most of the Winlogbeat jobs to work.

---

<div class="post-metadata">

**Author:** ![TheHunter1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thehunter1/32/80190_2.png) [@TheHunter1](https://discuss.elastic.co/u/TheHunter1)\
**Post date:** [January 5, 2021, 4:49pm UTC](https://discuss.elastic.co/t/ml-datafeed-lookback-retrieved-no-data/260104/9 "2021-01-05T16:49:26Z")

</div>

So I have just tried to install sysmon in one machine by running the command `.\Sysmon.exe -i -n -accepteula` and now I am seeing `Process Create (rule: ProcessCreate)` in the `event.action` field.

Tomorrow morning I will install `sysmon` in all my windows servers, and then I will try again to run the machine Learning jobs and I will keep you updated of the results

Thanks for your answers 😁

---

<div class="post-metadata">

**Author:** ![TheHunter1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thehunter1/32/80190_2.png) [@TheHunter1](https://discuss.elastic.co/u/TheHunter1)\
**Post date:** [January 6, 2021, 2:22pm UTC](https://discuss.elastic.co/t/ml-datafeed-lookback-retrieved-no-data/260104/10 "2021-01-06T14:22:02Z")

</div>

Hello again,

So after Installing `sysmon`, most of the winlogbeat machine Learning jobs are working, (8 are working, and still have problem in 3 jobs)

the jobs which are not working now are:

```auto
windows_anomalous_script
windows_rare_metadata_process
windows_rare_metadata_user

```

The query of the `windows_anomalous_script` job is:

```auto
GET _search
{
  "query": {
    "bool":{
      "filter":[{
        "term":{
          "winlog.channel":"Microsoft-Windows-PowerShell/Operational"
        }
      }]
    }
  }
}

```

and for the 2 others jobs is:

```auto
GET _search
{
  "query": {
    	"bool":{
    	  "filter":[{
    	    "term":{
    	      "destination.ip":"169.254.169.254"
    	    }
    	  }]
    	}
  }
}

```

Could you please tell me why I am not getting results for the first job, and why in 2 the other jobs the query is searching just for that IP address ??

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/1/6/16f861133bf12e67dbc487fca238505f1a15ad71.png)  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/d/e/de7646b8660e5c1e5561b44ba7d5197ed7aea13b.png)  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/e/f/ef418a518eeedf0e3193624bd04b1e1266dc8969.png)

---

<div class="post-metadata">

**Author:** ![droberts195](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/droberts195/32/17692_2.png) [@droberts195](https://discuss.elastic.co/u/droberts195)\
**Post date:** [January 6, 2021, 2:46pm UTC](https://discuss.elastic.co/t/ml-datafeed-lookback-retrieved-no-data/260104/11 "2021-01-06T14:46:13Z")

</div>

> [@TheHunter1](#):
>
> `Microsoft-Windows-PowerShell/Operational`

This relates to PowerShell scripts. If you don't use PowerShell scripts it's not surprising you don't have any events containing this field.

> `169.254.169.254`

This is the address of the metadata service that's available in Azure and AWS - see [Azure Instance Metadata Service for virtual machines - Azure Virtual Machines | Microsoft Learn](https://docs.microsoft.com/en-us/azure/virtual-machines/windows/instance-metadata-service) and [Retrieve instance metadata - Amazon Elastic Compute Cloud](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html). If your servers are not in Azure or AWS then, again, it's not surprising that you don't have events that contain this IP address.

I can see that we should try to make the user experience nicer when there is a set of ML jobs and it's expected that some of them won't have any input data and others will. I will take this up internally. For the time being I would suggest you just stop those jobs that aren't relevant to your environment so that they don't use up any resources. Just leave the 5 open that are relevant to you.

---

<div class="post-metadata">

**Author:** ![TheHunter1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thehunter1/32/80190_2.png) [@TheHunter1](https://discuss.elastic.co/u/TheHunter1)\
**Post date:** [January 6, 2021, 2:50pm UTC](https://discuss.elastic.co/t/ml-datafeed-lookback-retrieved-no-data/260104/12 "2021-01-06T14:50:41Z")

</div>

Thanks a lot for all your answers and explanations , it means that everything is working fine 😁

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 3, 2021, 2:50pm UTC](https://discuss.elastic.co/t/ml-datafeed-lookback-retrieved-no-data/260104/13 "2021-02-03T14:50:47Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.