# Packetbeat Rare DNS Questions ML Job Customization **URL:** \ **Category:** SIEM\ **Tags:** elastic-stack-machine-learning\ **Created:** [September 4, 2020, 1:44pm UTC](https://discuss.elastic.co/t/packetbeat-rare-dns-questions-ml-job-customization/247548 "2020-09-04T13:44:26Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![willemdh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/willemdh/32/16922_2.png) [@willemdh](https://discuss.elastic.co/u/willemdh)\ **Post date:** [September 4, 2020, 1:44pm UTC](https://discuss.elastic.co/t/packetbeat-rare-dns-questions-ml-job-customization/247548/1 "2020-09-04T13:44:26Z")
Hello, I have an issue with the packetbeat rare dns question ml job, which generates quite a bit of anomalies due to the fact that our hosts are frequently contacting `*.avqs.mcafee.com` url's, which have a random part. For example: `8m6-0.13-0.800.17d4.25eb.27e3.0.0.166z6e233qvcvbebe5f2wi76.avqs.mcafee.com` These anomalies are picked up by SIEM and as a SIEM ML Detection has nu way to filter stuff: ![image](https://us1.discourse-cdn.com/elastic/original/3X/0/5/05f3e075ea798cf4d1b9818904fa0fc350fb6dac.png) I will need to tune or filter the ml job itself. The query used in the ml job is: `{"bool":{"filter":[{"term":{"event.dataset":"dns"}},{"term":{"agent.type":"packetbeat"}}],"must_not":[{"bool":{"filter":{"term":{"dns.question.type":"PTR"}}}}]}}` So I'd like to discuss what would the best long term and flexible solution, so I can exclude certain domains when needed, without having to rebuild the ml job. Some possible solutions: - I could filter out `*.avqs.mcafee.com` in `dns.question.name` in the ml datafeed query - Even better (so I don't have to use expensive leading wildcard query) I could filter out `mcafee.com` in `dns.question.registered_domain` But both above options would require me to stop the datafeed, job and then update the datafeed query, which is not really user-friendly. Ideally I'd love to use a whitelist filter list like this: ![image](https://us1.discourse-cdn.com/elastic/original/3X/3/e/3ec0ac05dc4bc26f9029656790ffa655868173c3.png) But `dns.question.registered_domain` is not an option to scope. Feedback to enable me to dynamically filter on `dns.question.registered_domain` is welcome. Or is my only option to update the datafeed query in the ml job? Grtz Willem ---
**Author:** ![Frank\_Hassanabad](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/frank_hassanabad/32/49255_2.png) [@Frank\_Hassanabad](https://discuss.elastic.co/u/Frank_Hassanabad)\ **Post date:** [September 8, 2020, 3:15pm UTC](https://discuss.elastic.co/t/packetbeat-rare-dns-questions-ml-job-customization/247548/2 "2020-09-08T15:15:27Z")
Have you used the Filter lists from machine learning under settings? That might help you out some with what you're trying to do. I haven't used it directly myself but I hear good things about it from others. It will filter those things out before the anomalies are produced though but to a lot of people that's what they're aiming for: ![Screen Shot 2020-09-08 at 9.12.11 AM](https://us1.discourse-cdn.com/elastic/original/3X/0/6/063375133a887b8d5ea6a9d0f3887c1a7e186f6a.png) ---
**Author:** ![willemdh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/willemdh/32/16922_2.png) [@willemdh](https://discuss.elastic.co/u/willemdh)\ **Post date:** [September 9, 2020, 2:37pm UTC](https://discuss.elastic.co/t/packetbeat-rare-dns-questions-ml-job-customization/247548/3 "2020-09-09T14:37:43Z")
Currently on a holiday, but I'll definitely investigate the filter lists capabilities further. Thanks ---
**Author:** ![willemdh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/willemdh/32/16922_2.png) [@willemdh](https://discuss.elastic.co/u/willemdh)\ **Post date:** [September 23, 2020, 7:49am UTC](https://discuss.elastic.co/t/packetbeat-rare-dns-questions-ml-job-customization/247548/4 "2020-09-23T07:49:39Z")
So I tried to use the filter list, but it doesn't seem to work as expected.. ![image](https://us1.discourse-cdn.com/elastic/original/3X/a/2/a288936c13aa9b4759483287ab07a15e2574f792.png) whitelist\_server\_domain contains ![image](https://us1.discourse-cdn.com/elastic/original/3X/a/d/ad9cf4373a1dd8c390383e7f84e6845d6381f367.png) But I still encounter anomalies with \*.mcafee.com url's... Am I missing something? ---
**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\ **Post date:** [September 24, 2020, 4:17pm UTC](https://discuss.elastic.co/t/packetbeat-rare-dns-questions-ml-job-customization/247548/5 "2020-09-24T16:17:55Z")
Looks like a good reason to open a support ticket ---
**Author:** ![willemdh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/willemdh/32/16922_2.png) [@willemdh](https://discuss.elastic.co/u/willemdh)\ **Post date:** [September 24, 2020, 8:50pm UTC](https://discuss.elastic.co/t/packetbeat-rare-dns-questions-ml-job-customization/247548/6 "2020-09-24T20:50:06Z")
@richcollier Ticket 00614098 has been created. Grtz ---
**Author:** ![willemdh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/willemdh/32/16922_2.png) [@willemdh](https://discuss.elastic.co/u/willemdh)\ **Post date:** [September 29, 2020, 10:31am UTC](https://discuss.elastic.co/t/packetbeat-rare-dns-questions-ml-job-customization/247548/7 "2020-09-29T10:31:11Z")
@richcollier Just an fyi, I stumbled on this =\> [https://github.com/elastic/elasticsearch/issues/62948](https://github.com/elastic/elasticsearch/issues/62948) After closing / reopening the job, it works. While working on this, I got some additional questions. Is it possible to configure a rule for an ml job before the ml job has been started? For example during creation time or while editing. I'm asking this, because I created a new job from scratch, trying to prevent internal url's and other known domains that should be whitelist to 'pollute' my ml model. Afaik this is not possible yet. Is this already on Elastic's to do? If not, should I make a GH issue for it? Greetings. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 27, 2020, 10:31am UTC](https://discuss.elastic.co/t/packetbeat-rare-dns-questions-ml-job-customization/247548/8 "2020-10-27T10:31:34Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.