I am trying to run the integrated jobs for winlogbeat data, and most of them are having an error Datafeed lookback retrieved no data.
I already had the same problem using Elasticsearch version 8.0.0 from code source, So I decided to install the version 7.10.1, and even the beats are 7.10.1, and still have the same error.
[request body.duration.start]: expected value of type [number] but got [null] is due to a bug in the UI that was fixed in https://github.com/elastic/kibana/pull/86114. That fix will be in 7.11.
But both that and the original error of Datafeed lookback retrieved no data suggest that the datafeeds for your Winlogbeat jobs have ended up configured with searches that are not finding your Winlogbeat data. If you expand one of the rows on your jobs list you'll be able to see the datafeed config, including the indices that it's searching and the query it's using. Check whether you would expect a search using them to find your Winlogbeat data.
Here is an example of the datafeed configuration of one of the jobs that are not working, and as it's shown in the picture bellow, it is searching data in the right indice winlogbeat-*,
OK it might be worth having a browse through your winlogbeat-* indices using the Discover app. Do you see events that contain something similar to, but not identical to Process Create (rule: ProcessCreate)? I am wondering whether that string varies a bit between different versions of Windows, or whether it gets localized. For example, do you see documents in winlogbeat-* that have Process Create translated into some other language?
You might have discovered a flaw where these jobs have been accidentally tailored to one specific version of Windows or to Windows running in the US English locale only or something like that.
I used discover app to browse into my winlogbeat-* indice and then Toggle the column event.Action to see what I am getting inside.
there was some result which are translated to French language, and some are in English, but I didn't find in both languages Process Create (rule: ProcessCreate)
here is a list of all what I get in event.Action field:
Therein lies the problem. It is not obvious at first, but jobs like "windows_anomalous_process_all_hosts_ecs" actually depend on the field event.action being a sysmon value "ProcessCreate" as opposed to the expected "created-process" that windows event id's give.
So it looks like what you are seeing is a problem that has been reported before but nothing has been done about it yet.
It seems that you have to enable Sysmon for most of the Winlogbeat jobs to work.
So I have just tried to install sysmon in one machine by running the command .\Sysmon.exe -i -n -accepteula and now I am seeing Process Create (rule: ProcessCreate) in the event.action field.
Tomorrow morning I will install sysmon in all my windows servers, and then I will try again to run the machine Learning jobs and I will keep you updated of the results
Could you please tell me why I am not getting results for the first job, and why in 2 the other jobs the query is searching just for that IP address ??
I can see that we should try to make the user experience nicer when there is a set of ML jobs and it's expected that some of them won't have any input data and others will. I will take this up internally. For the time being I would suggest you just stop those jobs that aren't relevant to your environment so that they don't use up any resources. Just leave the 5 open that are relevant to you.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.