ML Job on Scripted field

Elastic Stack Elasticsearch
14

Hello Sarvendra,

Yes, I had suspected there was a syntax error, which is why I asked you to run the query and show me the output. The example query I posted was a copy/paste of the script that you posted - in other words, I didn't introduce the syntax error. Again, I wanted to see the OUTPUT of the query to see what the calculated value of the scripted field looks like - because I had suspected the syntax error.

Anyway, once you verified that your syntax of your scripted field is correct, then you can start to build your ML job. In your case, because of the scripted field definition in the ML job's datafeed, it might be easiest to do this via the Dev Tools Console:

First, define the job:

PUT _xpack/ml/anomaly_detectors/my_job
{
      "analysis_config": {
        "bucket_span": "5m",
        "detectors": [
            {
            "detector_description": "high_mean(Scripted_field)",
            "function": "high_mean",
            "field_name": "Scripted_field"
          }
        ],
        "influencers": [ ]
      },
      "data_description": {
        "time_field": "@timestamp"
      }
}

Then, define the datafeed:

PUT _xpack/ml/datafeeds/datafeed-my_job/
{
  "job_id": "my_job",
  "indices": [
    "master_neo"
  ],
      "query": {
        "match_all": {
          "boost": 1
        }
      },
      "script_fields": {
        "Scripted_field": {
          "script": {
            "inline":"(doc['numarticles'].sum()0.5)(-doc['avgtone'].avg() * 0.1)*(doc['goldsteinscale'].avg() * 10) * 0.0001",
"lang": "expression"
          },
          "ignore_failure": false
        }
      }
}

(Again, I suspect you're still missing a * between sum() and 0.5 but you need to validate this)

Then finally, preview the datafeed to make sure the that everything looks right.

GET _xpack/ml/datafeeds/datafeed-my_job/_preview/

It SHOULD look something like:

[
  {
    "@timestamp": 1486425600000,
    "Scripted_field": 264.4092102050781
  },
  {
    "@timestamp": 1486425600000,
    "Scripted_field": 1980.9256591796875
  },
...

(I'm just pretending I know the possible values of Scripted_field above)

Good luck!