Issue with: Inference usage to ingest index pattern to trained and tested model

Elastic Stack Kibana
1

I am trying to predict a field after training and testing a model. I believe since I've created a Data Analytics Job to do this, I will be needing inference.
A similar project has been issued in this following article.

https://www.elastic.co/blog/train-evaluate-monitor-infer-end-to-end-machine-learning-in-elastic

I followed the exact steps from the article and now I have a trained and tested model that i would like to ingest a csv file with a list of domains that I would like to have a predicted label given to (as a result from the learning the machine has done from the model)

I am stuck at the step under "Deploying our supervised model to enrich data at ingest time"
(I believe this is written into the Kibana DevTools console)

In the example given it is mentioned:

POST main_preds/_doc?pipeline=malware-classification
{
the author included the file details they wanted to label
}

Instead of the file details, I would like it to use an index pattern I already have with unlabeled names so that I label them according to the predictions done by the machine from the trained and tested model I have.

I would really appreciate your help with this. I am a beginner in Kibana and I will be very grateful if you could help me with this.

Best regards

2

Hi @rtest_9!
Thanks for your question!
If the data you want to classify using your malware detection model lives in an index / index pattern, you can re-index the data into a new index and pass the data through the ingest pipeline that contains your trained malware classification model. The reindex REST API call would look something like the below

POST _reindex
{
  "source": {
    "index": "unlabeled_dga_domains"
  },
  "dest": {
    "index": "dga_ngram_final",
    "pipeline": "malware-classification"
  }
}

However, if you are using the default scripts provided with the blog post you have to make sure that there is a field dns.question.registered_domain in your source index, because that is what the Painless script included with the blogpost is expecting.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.