How to compare 2 number fields in the same document

alerting

(Philip Romero) #1

I'm trying to set up what I feel should be a very simple watcher process, but I am new to building queries and having a very hard time. I'd like to compare the "actual" value to the "typical" value from a machine learning "record". Both fields are type double (numbers).

I can't seem to find any examples of number fields being compared to build from. In my final watcher process I'd like to get an alert when the "actual" value is 4x or greater than the "typical" value. Any pointers would be greatly appreciated.

Here's output for the index being used for background on the query. Thanks.

".ml-anomalies-custom-k12_ddos_unique_src_networks_5min_avg": {
"aliases": {
".ml-anomalies-.write-k12_ddos_unique_src_networks_5min_avg": {},
".ml-anomalies-k12_ddos_unique_src_networks_5min_avg": {
"filter": {
"term": {
"job_id": {
"value": "k12_ddos_unique_src_networks_5min_avg",
"boost": 1
}
}
}
}
},
"mappings": {
"doc": {
"_meta": {
"version": "6.4.2"
},
"dynamic_templates": [
{
"strings_as_keywords": {
"match": "*",
"mapping": {
"type": "keyword"
}
}
}
],
"properties": {
"actual": {
"type": "double"
},
"typical": {
"type": "double"
}
}
}
},
"settings": {
"index": {
"number_of_shards": "5",
"translog": {
"durability": "async"
},
"auto_expand_replicas": "0-1",
"provided_name": ".ml-anomalies-custom-k12_ddos_unique_src_networks_5min_avg",
"query": {
"default_field": "all_field_values"
},
"creation_date": "1523485305115",
"unassigned": {
"node_left": {
"delayed_timeout": "1m"
}
},
"number_of_replicas": "1",
"uuid": "PUVrI9QVQf-sutll3wo6FQ",
"version": {
"created": "6020399"
}
}
}
}


(Alexander Reelsen) #2

hey,

the above output has nothing to do with a query, this looks like an index settings response.

You could use a script field for this, but be aware that these calculations need to be done for every hit being returned, so this might be slow. You could try to save the calculation of this at index time using an ingest processor and thus have faster queries, if the factor is static.


(Philip Romero) #3

Correct, the output shown is the output from a GET /_indexname/, where _indexname was ".ml-anomalies-custom-k12_ddos_unique_src_networks_5min_avg". I wanted to show the index and fields of interest of my inquiry. As stated, I am very new to this environment and am having difficulty with the syntax for the query I'm trying to build.

I'm trying to get to an end state where the "actual" field is evaluated against the "typical" field (x4) to get a true/false response on whether the "actual" field is >= (4 * "typical"). I want to run the query against the this specific index only with documents of "result_type: "record"", which is very small. It should run fast since there are only a handful of documents in it.


(system) #4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.