Rollup job - should and minimum_should_match not working

When querying rollup jobs minimum_should_match is not working (v7.17) making should clauses useless.

Is there a fix/workaround for this?

The rollup job limitations page lists compound queries explicitly as allowed ones, so in theory it should work.

I may not be able to directly help here, but it'd be useful if you showed your rollup job, the output you are getting, and how that differs from what you expect.

Thank you for taking the time to answer.

I've created a paste here: https://pastebin.com/6bJPp8mT that creates a sample index (cardata, but it has totally bogus data), puts 5 docs into it then creates and starts a rollup job. (It also cleans up after itself (L41-43).)

Should I file a bug in GitHub? Or am I doing something wrong? :slight_smile:

A ‘should’ clause clause is only mandatory if it is on its own in a ‘bool’ clause. When it is alongside a must/filter/must_not it is relegated to an optional clause that gives extra points to scores of docs matching the other mandatory clauses.
To make a should act as a mandatory set of ORed choices it can often be necessary to wrap it in a bool container on its own and then embed that as a must inside the main bool query.

It seems to me that with an extra bool, the result is the same: the 'normal' search works, the rollup search does not.

Using the dataset from my pastebin link above, this 'normal' query returns GBP and amount 17.

If you replace the index name to rollup-cardata and the API endpoint to _rollup_search, you get 0 hits. (Interestingly, if I leave should clause as a 2nd filter (on the same level as the term:currency:GBP), I get the erroneous operation I mentioned in the op (GBP and 82 as amount).

GET cardata/_search?size=0
{
  "query": {
    "bool": {
      "filter": [
        {
          "term": {
            "currency": "GBP"
          }
        },
        {
          "bool": {
            "should": [
              {
                "term": {
                  "car.original_state": "CLEAN"
                }
              },
              {
                "term": {
                  "car.original_state": "UPGRADED"
                }
              }
            ],
            "minimum_should_match": 1
          }
        }
      ]
    }
  },
  "aggs": {
    "currency_agg": {
      "terms": {
        "field": "currency"
      },
      "aggs": {
        "amount_agg": {
          "sum": {
            "field": "amount"
          }
        }
      }
    }
  }
}

I’ll leave it to others to diagnose why the roll up query may not be working in this case but the comment on the bool/should logic still stands.

1 Like