# Ngram behavior vs wildcard field type

**URL:** <https://discuss.elastic.co/t/ngram-behavior-vs-wildcard-field-type/302762>\
**Category:** Elasticsearch\
**Created:** [April 20, 2022, 2:49am UTC](https://discuss.elastic.co/t/ngram-behavior-vs-wildcard-field-type/302762 "2022-04-20T02:49:16Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![maratusa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maratusa/32/100500_2.png) [@maratusa](https://discuss.elastic.co/u/maratusa)\
**Post date:** [April 20, 2022, 2:49am UTC](https://discuss.elastic.co/t/ngram-behavior-vs-wildcard-field-type/302762/1 "2022-04-20T02:49:16Z")

</div>

Hi,

Please help me understand n-gram and wildcard field type behaviors. I am working on an application that offers a search by phone number. It will be a contain search. e.g. search for phone number containing "234890".  
Our Elasticsearch index has close to 1 billion documents.

While looking into options, I came across [wildcard field type](https://www.elastic.co/guide/en/elasticsearch/reference/master/keyword.html#wildcard-field-type) which seems to fit our use case. We haven't done any performance tests yet.

wildcard field type uses 3-gram - so I wanted to test 3-gram and wildcard, but I have hard time understanding why my example below does not match my expectations.

Here is the example I am using:

```auto
PUT ngram-index
{
  "settings": {
    "index": {
      "number_of_shards": "2",
      "number_of_replicas": "1"
    },
    "analysis": {
      "analyzer": {
        "ngram": {
          "tokenizer": "ngram"
        }
      },
      "tokenizer": {
        "ngram": {
          "type": "ngram",
          "min_gram": 3,
          "max_gram": 3
        }
      }
    }
  },
  "mappings": {
    "dynamic": "strict",
    "properties": {
      "ngram_field": {
        "type": "text",
        "analyzer": "ngram"
      },
      "wildcard_field": {
        "type": "wildcard",
        "ignore_above": 25
      }
    }
  }
}

PUT /ngram-index/_doc/1
{
  "ngram_field": "1234567",
  "wildcard_field": "1234567"
}

PUT /ngram-index/_doc/2
{
  "ngram_field": "234890",
  "wildcard_field": "234890"
}

```

When I ran a search using the wildcard field I get the right results per my expectations which is document id=2

```auto
POST /ngram-index/_search
{
  "query": {
    "bool": {
      "filter": [
        {
          "wildcard": {
            "wildcard_field": {
              "value": "23489*"
            }
          }
        }
      ]
    }
  }
}

```

however when I search using 3-gram - I get no results. I was expecting both documents id=1 and id=2 to be returned because the 3-gram "234" exist in both documents.  
can you please help me understand this behavior:

```auto
POST /ngram-index/_search
{
  "query": {
    "bool": {
      "filter": [
        {
          "wildcard": {
            "ngram_field": {
              "value": "23489*"
            }
          }
        }
      ]
    }
  }
}

```

Thanks,  
Moulay

---

<div class="post-metadata">

**Author:** ![RabBit\_BR](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rabbit_br/32/82261_2.png) [@RabBit\_BR](https://discuss.elastic.co/u/RabBit_BR)\
**Post date:** [April 20, 2022, 8:53pm UTC](https://discuss.elastic.co/t/ngram-behavior-vs-wildcard-field-type/302762/2 "2022-04-20T20:53:13Z")

</div>

Hi!

This happens because of your ngram.  
The tokens for 234890 -\> 234, 348, 489, 890

Its pattern is 23489\*, the wildcard will not find a match because it doesn't have token 23489.

If you test with 234\* or 23\*, it will retrieve documents.

---

<div class="post-metadata">

**Author:** ![maratusa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maratusa/32/100500_2.png) [@maratusa](https://discuss.elastic.co/u/maratusa)\
**Post date:** [April 21, 2022, 12:41am UTC](https://discuss.elastic.co/t/ngram-behavior-vs-wildcard-field-type/302762/3 "2022-04-21T00:41:26Z")

</div>

I thought that the ngram analyzer - at both **search** and **index** times - will token 234890 into 234, 348, 489, 890 - hence I was expecting results. If that's not the case? how can I set up my search\_analyzer to behave similarly to the index\_analyzer?

One more thing that I am trying to understand, is why the wildcard field type is returning the correct results (when searching for 23489\*)? should it behave similarly to the ngram analyzer as it's based on 3-gram tokens as well?

---

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [April 21, 2022, 7:14am UTC](https://discuss.elastic.co/t/ngram-behavior-vs-wildcard-field-type/302762/4 "2022-04-21T07:14:02Z")

</div>

The wildcard field has a ton of code in it to make wildcard and regex queries work as expected.  
It first executes an approximation phase using the ngram index to accelerate queries (but only where appropriate) and then feeds candidate matches into a second validation phase that checks the wildcard/regex works on the original full doc value. Great care is taken to ensure the approximation phase uses an ngram query that eliminates as much as possible but without causing false negatives.

Just using an ngram index is not a substitute for this logic.

---

<div class="post-metadata">

**Author:** ![anime\_lover](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anime_lover/32/100265_2.png) [@anime\_lover](https://discuss.elastic.co/u/anime_lover)\
**Post date:** [April 21, 2022, 11:10am UTC](https://discuss.elastic.co/t/ngram-behavior-vs-wildcard-field-type/302762/5 "2022-04-21T11:10:28Z")

</div>

Hi there ,  
the thing i understood is that you need a query to search phone number containing certain queried value ,right?

if yes then you can search using query\_string  
eg:

```auto
GET /index/_search
{
  "query": {
    "query_string": {
      "default_field": "mobile",
      "query": "*05793*"
    }
  }
}

```

I hope it might help you

---

<div class="post-metadata">

**Author:** ![maratusa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maratusa/32/100500_2.png) [@maratusa](https://discuss.elastic.co/u/maratusa)\
**Post date:** [April 23, 2022, 12:44am UTC](https://discuss.elastic.co/t/ngram-behavior-vs-wildcard-field-type/302762/6 "2022-04-23T00:44:18Z")

</div>

Can you help me understand why ngram are not tokenizing the search value at query time?  
I was expecting my query to search for 234, 348, 489, 89\*.

```auto
POST /ngram-index/_search
{
  "query": {
    "bool": {
      "filter": [
        {
          "wildcard": {
            "ngram_field": {
              "value": "23489*"
            }
          }
        }
      ]
    }
  }
}

```

---

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [April 23, 2022, 4:03pm UTC](https://discuss.elastic.co/t/ngram-behavior-vs-wildcard-field-type/302762/7 "2022-04-23T16:03:55Z")

</div>

Wildcard queries are part of [the term-level](https://www.elastic.co/guide/en/elasticsearch/reference/current/term-level-queries.html) family of queries whose docs state:

> “Unlike full-text queries, term-level queries do not analyze search terms. Instead, term-level queries match the exact terms stored in a field”

---

<div class="post-metadata">

**Author:** ![maratusa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maratusa/32/100500_2.png) [@maratusa](https://discuss.elastic.co/u/maratusa)\
**Post date:** [April 27, 2022, 4:45pm UTC](https://discuss.elastic.co/t/ngram-behavior-vs-wildcard-field-type/302762/8 "2022-04-27T16:45:33Z")

</div>

Thanks Mark.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 25, 2022, 4:46pm UTC](https://discuss.elastic.co/t/ngram-behavior-vs-wildcard-field-type/302762/9 "2022-05-25T16:46:29Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.