Continuous Transform query, does it update?

Elastic Stack Elasticsearch
1

If you use a query when creating a continuous transform will it only ever work on the query subset that is created when the transform is started?

{
  "bool": {
    "should": [
      {
        "match": {
          "state": "state_1"
        }
      },
      {
        "match": {
          "state": "state_2"
        }
      }
    ],
    "minimum_should_match": 1
  }
}

I have a transform that uses a query to initially filter some of my data by a 'state' field (e.g values of state_1, state_2 etc.) before creating the aggregation, the state can change e.g from state_3 to state_2 but this data is not in the transform.

If I delete the transform and start it again it will include the updated data, is this how a transform is supposed to work?

2

Hi!

The update transform API enables you to change a subset of the configuration details, including the query. For example:

POST _transform/test1/_update
{
  "source" : {
        "index" : [
          "myIndices"
        ],
        "query" : {
          "bool": {
            "should": [
              {
                "match": {
                  "state": "state_3"
                }
              },
              {
                "match": {
                  "state": "state_2"
                }
              }
              ],
              "minimum_should_match": 1
          }
        }
  }
}
3

Does Transform limitations | Elasticsearch Guide [7.15] | Elastic help clarify things at all?

4

Each time the continuous transform is refreshed is the query run again on all data or is the query only run once when the transform is started to create an initial subset?

5

It's the former. For continuous transforms, it's part of the process where the transform is checking for changes to source indices. There's some information about these checkpoints in the documentation:

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.