How to efficiently determine which field is behind a search hit


(Mahdi) #1

Hello,

We allow our customers to create custom properties which are indexed in elasticsearch. We do not know in advance what these custom properties are and we would like to be able to auto-suggest when a user types a search request what properties (ie, fields) are relevant to the search. For example, a customer may create a field "order_status" that indicates whether an order is "PENDING", "SHIPPED", "DELIVERED", etc.. If the user searches for "PENDING", we would like to suggest "order_status" as property that can be used for filtering.

The way we do this today is to run a first query to determine property names (we have a field that contains property values that we aggregate on to determine the complete list). We then run aggregations with a match query on the properties that we think are relevant. This is inefficient and slow most of the times.

  1. is there a way to request elasticsearch to search all fields of an index and as part of the result, indicate for each document which field caused the hit to happen ? One solution is to determine all fields (the list is dynamic), then send a query against the list of those fields with highlighting enabled. That seems like an overkill since we only care about the field names. We could also have a large number of properties, so the query would be abnormally big (not sure if that would be a problem for ES). I was wondering if there was a better way of doing this.

  2. is there an efficient way to determine the list of all fields based on a query or a filter ?

Thanks.


(system) #2