5.0: can't have Terms query within a Filter Aggregation?


(Sean Mahan) #1

I have an application with a social stream feature a la Twitter. Users can "subscribe" to email notifications for people mentioned in posts, but that subscription doesn't affect what appears in a user's stream. So to display the stream, I need to:

  1. Fetch the posts users should see
  • Figure out which of those posts the user is set to receive emails about, to display a "toggle email notifications" button in the UI.

I don't want to store a list of subscribed users with each post, so instead there's an index of users that includes an array of subscribed-to people. In Elasticsearch 2.x, that let me accomplished the second task by doing a filter aggregation to see which posts, out of the ones due to be displayed, mentioned people that the user was subscribed to email notifications for. Then I'd do a terms aggregation to just get postIDs, which I'd use to to toggle that email subscription button.

Post mapping:

"post": {
  "properties": {
      "mentionIDs": {
            "type": "keyword"
       }
  }
}

User mapping:

"user": {
  "properties": {
     "notifications": {
       "type": "keyword"
     }
  }
}

And here's the tail end of my query against "items":

"aggs": {
  "itemsNotify": {
    "filter": {
      "terms": {
        "mentionIDs": {
          "index": "users",
          "type": "user",
          "id": "USR|CLIENTID|1234",
          "path": "notifications",
          "routing": "CLIENTID"
        }
      }
    },
    "aggs": {
      "notifyIDs": {
        "terms": {
          "field": "postID"
        }
      }
    }
   }
  }

This no longer works in Elasticsearch 5.0, which instead returns the error:

{
  "error": {
    "root_cause": [
      {
        "type": "unsupported_operation_exception",
        "reason": "unsupported_operation_exception: query must be rewritten first"
      },
      {
        "type": "unsupported_operation_exception",
        "reason": "query must be rewritten first"
      }
    ],
}

I'll admit that I don't understand that error. Is this kind of Terms Query (referencing terms stored in a separate index) within a Filter Aggregation no longer supported? Is there a better fallback strategy for me than just running a second query that takes the original and adds that terms query as a filter?


(Adrien Grand) #2

Thanks for reporting this issue, I opened https://github.com/elastic/elasticsearch/issues/21301.


(Sean Mahan) #3

Thanks! I've subscribed to the issue.


(system) #4