Export forecast data

[Oscarlf (OSCAR LIMA)]

Hi everyone!

Is there any possibility of exporting the data reflected in a forecast scenario? I can see it and the comparison in the Single Metric Viewer, but I would like to use it or manage it to do some other calculations, since it was the purpose to forecast that data, in my case capacity planning, for example.

Thanks in advance!

OL

[Hendrik_Muhs]

Hi Oscar,

all forecast results are written back to the result index of the job, that is default the shared index: .ml-anomalies-shared but you can also use a dedicated index.

Therefore you can access all results via search, e.g.

GET .ml-anomalies-shared/_search
{
  "query": {
    "bool": {
      "filter": [
        {
          "query_string": {
            "query": "result_type:model_forecast"
          }
        },
        {
          "query_string": {
            "query": "job_id:n1"
          }
        }
      ]
    }
  }
}

In the result you will see the fields forecast_lower, forecast_upper, forecast_prediction along with timestamp which is the timestamp of the prediction.

Happy forecasting!

[Oscarlf (OSCAR LIMA)]

Hi Hendrik_Muhs, so helpful your support, Thanks a lot.

But you kow, I'm wondering if there is a chance to export this data, I mean, like when we are in a "Visualize" table and there is an option to export in "raw" or "formatted". The think is that we are trying to use all the "forectast_prediction" data to do some other analysis like capacity requirements, etc.

Besides, when we run a forecast scenario on the job and it has more than one "detector", how could I filter the one I want to see? Running the query you suggested it appears a "detector_index" followed by a number but doesn's seems to have a relation or can't find it yet. Could you please help with this?

Thanks in advance.

[Hendrik_Muhs]

Hi Oscar,

sorry for the late reply. As said exporting data can be done via queries and some scripting around it. Having that said, there is also the possibility to use the kibana UI to export the data using "Visualize", e.g. a table view. The 1st step to be done is making the data visible for kibana by creating an index pattern, via management/kibana/index patterns/create index pattern. Check include system indices and create an index pattern for .ml-anomalies-shared or .ml-anomalies-JOBID if you use a dedicated index. That should make the data visible for all kibana visualizations. As 2nd step you can now create a table visualization using the fields of interest e.g. forecast_prediction as metric and a data histogram for the buckets with the granularity of your choice. The data can now be exported using Export.

Regarding your 2nd question about selecting the detector: As detector index is a number a match query should do the trick, for example:

GET .ml-anomalies-shared/_search
{
  "query": {
    "bool": {
      "filter": [
        {
          "query_string": {
            "query": "result_type:model_forecast"
          }
        },
        {
          "query_string": {
            "query": "job_id:j1"
          }
        },
        {
          "match": {
            "detector_index": "2"
          }
        }
      ]
    }
  }
}