# Marking the missing words in search results

**URL:** <https://discuss.elastic.co/t/marking-the-missing-words-in-search-results/66986>\
**Category:** Elasticsearch\
**Created:** [November 23, 2016, 2:13pm UTC](https://discuss.elastic.co/t/marking-the-missing-words-in-search-results/66986 "2016-11-23T14:13:46Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![itasyurt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/itasyurt/32/45136_2.png) [@itasyurt](https://discuss.elastic.co/u/itasyurt)\
**Post date:** [November 23, 2016, 2:13pm UTC](https://discuss.elastic.co/t/marking-the-missing-words-in-search-results/66986/1 "2016-11-23T14:13:46Z")

</div>

Hello,  
I want to implement a feature to show missing search terms in search results similar to Google's.  
Is there any proper way to do in Elasticsearch. Current Elasticsearch version we are using is 1.5.2.

Thank you.

An example from Google is below, where "potatoes" is missing.  
 ![](https://us1.discourse-cdn.com/elastic/original/2X/7/7532da8b1af582f6a8b9b079057d737a7739b0ba.png)

---

<div class="post-metadata">

**Author:** ![mainec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mainec/32/5557_2.png) [@mainec](https://discuss.elastic.co/u/mainec)\
**Post date:** [November 23, 2016, 2:18pm UTC](https://discuss.elastic.co/t/marking-the-missing-words-in-search-results/66986/2 "2016-11-23T14:18:52Z")

</div>

You should have a look at the match query, in particular the `minimum_should_match` option:

[https://www.elastic.co/guide/en/elasticsearch/reference/5.x/query-dsl-match-query.html](https://www.elastic.co/guide/en/elasticsearch/reference/5.x/query-dsl-match-query.html)

By the way - what's the reason for still being on 1.5.2?

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [November 23, 2016, 2:41pm UTC](https://discuss.elastic.co/t/marking-the-missing-words-in-search-results/66986/3 "2016-11-23T14:41:31Z")

</div>

The only thing I can think of is to do the following on the client:

Let's say you search for `elasticsearch potatoes`:

- Split the text (you can use `_analyze` endpoint for that)
- For each generated token, here: `elasticsearch`, `potatoes`, generate a should clause using named queries.
- As a response, you will get for each doc, the list of queries which matched,. So you can on client side know which queries did not match.

# Full example

**Tested on 5.0.1**

## Index some data

```auto
DELETE index
POST index/doc
{
  "content": "elasticsearch you know for search"
}
POST index/doc
{
  "content": "potatoes you know for lunch"
}

```

## Analyze the text

```auto
GET _analyze
{
  "text": "elasticsearch potatoes"
}

```

it gives back 2 tokens:

```auto
{
  "tokens": [
    {
      "token": "elasticsearch",
      "start_offset": 0,
      "end_offset": 13,
      "type": "<ALPHANUM>",
      "position": 0
    },
    {
      "token": "potatoes",
      "start_offset": 14,
      "end_offset": 22,
      "type": "<ALPHANUM>",
      "position": 1
    }
  ]
}

```

## Create the search

Iterate over the tokens and create the should clause for them:

```auto
GET index/_search
{
  "query": {
    "bool": {
      "should": [
        {
          "match": {
            "content": {
              "query": "elasticsearch",
              "_name": "elasticsearch"
            }
          }
        },
        {
          "match": {
            "content": {
              "query": "potatoes",
              "_name": "potatoes"
            }
          }
        }
      ]
    }
  }
}

```

It gives:

```auto
{
  "took": 4,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 2,
    "max_score": 0.2824934,
    "hits": [
      {
        "_index": "index",
        "_type": "doc",
        "_id": "AViRm-3jWVDw7QDfjwKn",
        "_score": 0.2824934,
        "_source": {
          "content": "elasticsearch you know for search"
        },
        "matched_queries": [
          "elasticsearch"
        ]
      },
      {
        "_index": "index",
        "_type": "doc",
        "_id": "AViRm_VGWVDw7QDfjwKo",
        "_score": 0.2824934,
        "_source": {
          "content": "potatoes you know for lunch"
        },
        "matched_queries": [
          "potatoes"
        ]
      }
    ]
  }
}

```

So you know that for the first doc, `potatoes` is missing and for the second doc `elasticsearch` is missing.

## Using a template

You can also use a template like this:

```auto
GET index/_search/template
{
  "inline": "{\"query\":{\"bool\":{\"should\":[{{#term}}{\"match\":{\"content\":{\"query\": \"{{.}}\",\"_name\": \"{{.}}\"}}},{{/term}}{}]}}}",
  "params": {
    "term": ["elasticsearch", "potatoes"]
  }
}

```

The template part is actually the following but it needs to be wrapped here in a string as it's not a valid JSON otherwise:

```auto
{
  "query": {
    "bool": {
      "should": [
        {{#term}}
        {
          "match": {
            "content": {
              "query": "{{.}}",
              "_name": "{{.}}"
            }
          }
        },
        {{/term}}
        {}
      ]
    }
  }
}

```

I hope this helps.

---

<div class="post-metadata">

**Author:** ![itasyurt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/itasyurt/32/45136_2.png) [@itasyurt](https://discuss.elastic.co/u/itasyurt)\
**Post date:** [November 23, 2016, 2:42pm UTC](https://discuss.elastic.co/t/marking-the-missing-words-in-search-results/66986/4 "2016-11-23T14:42:08Z")

</div>

Thank you,  
We're using 1.5.2 since the system is already live and we have some other blockers (plugin migration etc.) before we upgrade to a new version.

Let me give more context that, I don't want excluding results matching less than some minimum, but mark the missing words for each field. You can think of it some sort of inverse of the highlighting.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [November 23, 2016, 3:06pm UTC](https://discuss.elastic.co/t/marking-the-missing-words-in-search-results/66986/5 "2016-11-23T15:06:43Z")

</div>

I just updated my answer to show you also how you can use search templates to make that even easier to use.

---

<div class="post-metadata">

**Author:** ![itasyurt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/itasyurt/32/45136_2.png) [@itasyurt](https://discuss.elastic.co/u/itasyurt)\
**Post date:** [November 23, 2016, 4:12pm UTC](https://discuss.elastic.co/t/marking-the-missing-words-in-search-results/66986/7 "2016-11-23T16:12:40Z")

</div>

Thank you very much,  
your answer and `matched_queries` looks like an option.  
Some concerns we have:

1. We use a custom analyzer with a number of synonyms like('javascript=\>js'), so we're going to need the original terms(before analysis)
2. Adding other clauses may affect our score calculation, but I think it's something that can be easily handled.

Btw, does implementing a server side solution(like implementing a plugin) make sense here or a bit too heavy for this use case?

Thank you for your help.  
ibrahim--

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [November 23, 2016, 4:47pm UTC](https://discuss.elastic.co/t/marking-the-missing-words-in-search-results/66986/8 "2016-11-23T16:47:49Z")

</div>

> [@itasyurt](#):
>
> We use a custom analyzer with a number of synonyms like('javascript=\>js'), so we're going to need the original terms(before analysis)

Yeah may be. Or use the same custom analyzer in `_analyze` API but then use `term` query instead of `match` query and you'll have the same behavior.

> [@itasyurt](#):
>
> Btw, does implementing a server side solution(like implementing a plugin) make sense here or a bit too heavy for this use case?

You can but the problem with plugins is that they are tight to elasticsearch version. So you will need to update it any time you upgrade elasticsearch. Using only the REST endpoint would be easier to maintain IMO.

---

<div class="post-metadata">

**Author:** ![itasyurt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/itasyurt/32/45136_2.png) [@itasyurt](https://discuss.elastic.co/u/itasyurt)\
**Post date:** [November 23, 2016, 5:41pm UTC](https://discuss.elastic.co/t/marking-the-missing-words-in-search-results/66986/9 "2016-11-23T17:41:03Z")

</div>

Thanks. I'll give a try.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 21, 2016, 5:41pm UTC](https://discuss.elastic.co/t/marking-the-missing-words-in-search-results/66986/10 "2016-12-21T17:41:03Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.