# Filter logs in ML Job

**URL:** <https://discuss.elastic.co/t/filter-logs-in-ml-job/179807>\
**Category:** Kibana\
**Tags:** elastic-stack-machine-learning\
**Created:** [May 6, 2019, 3:45pm UTC](https://discuss.elastic.co/t/filter-logs-in-ml-job/179807 "2019-05-06T15:45:38Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jesus\_Rodrigo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jesus_rodrigo/32/43168_2.png) [@Jesus\_Rodrigo](https://discuss.elastic.co/u/Jesus_Rodrigo)\
**Post date:** [May 6, 2019, 3:45pm UTC](https://discuss.elastic.co/t/filter-logs-in-ml-job/179807/1 "2019-05-06T15:45:38Z")

</div>

Hello,

I'm trying to do a Job Machine Learning but I would need to the logs for a field that I created with a index script. The problem is that I can only access the fields defined in the index, but not those created. How could I access these fields to filter by them?

---

<div class="post-metadata">

**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\
**Post date:** [May 6, 2019, 6:02pm UTC](https://discuss.elastic.co/t/filter-logs-in-ml-job/179807/2 "2019-05-06T18:02:11Z")

</div>

Hello Jesus,

I'm having a little trouble understanding what you mean by "field that I created with a index script". Are you referring to [script fields](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-script-fields.html)?

---

<div class="post-metadata">

**Author:** ![Jesus\_Rodrigo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jesus_rodrigo/32/43168_2.png) [@Jesus\_Rodrigo](https://discuss.elastic.co/u/Jesus_Rodrigo)\
**Post date:** [May 7, 2019, 7:40am UTC](https://discuss.elastic.co/t/filter-logs-in-ml-job/179807/3 "2019-05-07T07:40:37Z")

</div>

Hi, thanks you for your reply.

I'm referring to script field that I calculate with script in index pattern. For example, with a date, I extract if the day is monday, tuesday, etc. And I will filter my logs and discard those that occurred on weekends

---

<div class="post-metadata">

**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\
**Post date:** [May 7, 2019, 11:40am UTC](https://discuss.elastic.co/t/filter-logs-in-ml-job/179807/4 "2019-05-07T11:40:40Z")

</div>

Ok, well, if you already have a working filter for a standard elasticsearch search, then the only thing that remains is getting an ML job to implement that search.

You have a few choices, you can either:

1. Use the Advanced Job configuration wizard and use the Datafeed tab to put in your custom query. For example:

![image](https://us1.discourse-cdn.com/elastic/original/3X/3/9/39e4edde10bf454a74396351d243c5eda41e5538.png)

1. Use the Advanced Job configuration wizard and use the JSON tab to put in your custom query:

![image](https://us1.discourse-cdn.com/elastic/original/3X/1/2/126b60ab2961bfb4eef7d110a61d7f9d81cab0a3.png)

1. Use the [datafeed API](https://www.elastic.co/guide/en/elasticsearch/reference/7.0/ml-put-datafeed.html) to set the custom query you want the ML job to use

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 4, 2019, 11:40am UTC](https://discuss.elastic.co/t/filter-logs-in-ml-job/179807/5 "2019-06-04T11:40:40Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.