# Filter on \_search performed with text\_expansion is not working

**URL:** <https://discuss.elastic.co/t/filter-on-search-performed-with-text-expansion-is-not-working/355110>\
**Category:** Elasticsearch\
**Created:** [March 11, 2024, 1:00am UTC](https://discuss.elastic.co/t/filter-on-search-performed-with-text-expansion-is-not-working/355110 "2024-03-11T01:00:16Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ankur\_Garg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ankur_garg/32/122097_2.png) [@Ankur\_Garg](https://discuss.elastic.co/u/Ankur_Garg)\
**Post date:** [March 11, 2024, 1:00am UTC](https://discuss.elastic.co/t/filter-on-search-performed-with-text-expansion-is-not-working/355110/1 "2024-03-11T01:00:16Z")

</div>

I have an Elasticsearch index ingested with an inference pipeline using ELSER. While performing the search I would like to filter and show all the data where my field "Country" has a certain value "XYZ".  
My query below works:

```auto
GET service-index/_search
{
  "query": {
    "bool": {
      "must": [{
        "text_expansion": {
          "ml.tokens": {
            "model_id": ".elser_model_2_linux-x86_64",
            "model_text": "Food tour"
          }
        }
      }]
    }
  },
  "fields": [
    "ServiceDescription"
  ], 
  "_source": false
}

```

But the below query does not work.

```auto
GET service-index/_search
{
  "query": {
    "bool": {
      "must": [{
        "text_expansion": {
          "ml.tokens": {
            "model_id": ".elser_model_2_linux-x86_64",
            "model_text": "Food tour"
          }
        }
      }],
      "filter": [{
        "term": {
          "Locale": "Paris"
        }
      }]
    }
  },
  "fields": [
    "ServiceDescription"
  ], 
  "_source": false
}

```

I have checked the index and Locale has a value of "Paris" for the documents that are returned. I also looked up question asked here: [How to filter \_search performed with text\_expansion](https://discuss.elastic.co/t/how-to-filter-search-performed-with-text-expansion/344646)

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [March 11, 2024, 2:05am UTC](https://discuss.elastic.co/t/filter-on-search-performed-with-text-expansion-is-not-working/355110/2 "2024-03-11T02:05:28Z")

</div>

Hi @Ankur_Garg, Welcome to the community.

I am not sure what your issue is as you did not show the data and mapping but perhaps this will help...

The Docs... [Here](https://www.elastic.co/guide/en/elasticsearch/reference/current/semantic-search-elser.html) and [Here](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-filter-context.html)

These work as expected

```auto
DELETE discuss-test-elser

PUT discuss-test-elser
{
  "mappings": {
    "properties": {
      "content_embedding": {
        "type": "sparse_vector"
      },
      "content": {
        "type": "text"
      },
      "region": {
        "type": "keyword"
      }
    }
  }
}

PUT _ingest/pipeline/elser-v2-test
{
  "processors": [
    {
      "inference": {
        "model_id": ".elser_model_2",
        "input_output": [ 
          {
            "input_field": "content",
            "output_field": "content_embedding"
          }
        ]
      }
    }
  ]
}

POST discuss-test-elser/_doc?pipeline=elser-v2-test
{
  "content" : "I had a great tour of the Notre Dame Church",
  "region" : "Paris"
}
  
POST discuss-test-elser/_doc?pipeline=elser-v2-test
{
  "content" : "I had a great tour of the El Domo",
  "region" : "Florence"
}  

POST discuss-test-elser/_doc?pipeline=elser-v2-test
{
  "content" : "I had dinner in Rome",
  "region" : "Rome"
} 

GET discuss-test-elser/_search
{
  "_source": [
    "content",
    "region"
  ]
}

# Search with no filter
GET discuss-test-elser/_search
{
  "_source": [
    "content",
    "region"
  ],
  "query": {
    "bool": {
      "must": [
        {
          "text_expansion": {
            "content_embedding": {
              "model_id": ".elser_model_2",
              "model_text": "great tour"
            }
          }
        }
      ]
    }
  }
}  

# Search with Filter performant, Filter is not scored 
GET discuss-test-elser/_search
{
  "_source": [
    "content",
    "region"
  ],
  "query": {
    "bool": {
      "must": [
        {
          "text_expansion": {
            "content_embedding": {
              "model_id": ".elser_model_2",
              "model_text": "great tour"
            }
          }
        }
      ],
      "filter": [
        {
          "term": {
            "region": "Paris"
          }
        }
      ]
    }
  }
}  

# Search with Must with 2 queries works slightly less performant 
GET discuss-test-elser/_search
{
  "_source": [
    "content",
    "region"
  ],
  "query": {
    "bool": {
      "must": [
        {
          "text_expansion": {
            "content_embedding": {
              "model_id": ".elser_model_2",
              "model_text": "great tour"
            }
          }
        },
        {
          "term": {
            "region": {
              "value": "Paris"
            }
          }
        }
      ]
    }
  }
}  
  
# Search with Should to show scoring
GET discuss-test-elser/_search
{
  "_source": [
    "content",
    "region"
  ],
  "query": {
    "bool": {
      "should": [
        {
          "text_expansion": {
            "content_embedding": {
              "model_id": ".elser_model_2",
              "model_text": "great tour"
            }
          }
        },
        {
          "term": {
            "region": {
              "value": "Paris"
            }
          }
        }
      ]
    }
  }
}  

```

---

<div class="post-metadata">

**Author:** ![Ankur\_Garg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ankur_garg/32/122097_2.png) [@Ankur\_Garg](https://discuss.elastic.co/u/Ankur_Garg)\
**Post date:** [March 11, 2024, 12:57pm UTC](https://discuss.elastic.co/t/filter-on-search-performed-with-text-expansion-is-not-working/355110/3 "2024-03-11T12:57:05Z")

</div>

Thanks @stephenb , let me take a look through this and reindex. Does it have to be "sparse vector" or can it be "rank features" too ?

---

<div class="post-metadata">

**Author:** ![Sean\_Story](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sean_story/32/69987_2.png) [@Sean\_Story](https://discuss.elastic.co/u/Sean_Story)\
**Post date:** [March 11, 2024, 1:14pm UTC](https://discuss.elastic.co/t/filter-on-search-performed-with-text-expansion-is-not-working/355110/4 "2024-03-11T13:14:19Z")

</div>

`sparse_vector` is the newer type, and was added with ELSER in mind. You can definitely still use `rank_features`, but that field type will also allow other types of queries against it than semantic text search, so it may be better to leverage `sparse_vector`.

---

<div class="post-metadata">

**Author:** ![Ankur\_Garg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ankur_garg/32/122097_2.png) [@Ankur\_Garg](https://discuss.elastic.co/u/Ankur_Garg)\
**Post date:** [March 11, 2024, 1:29pm UTC](https://discuss.elastic.co/t/filter-on-search-performed-with-text-expansion-is-not-working/355110/5 "2024-03-11T13:29:53Z")

</div>

Thanks @Sean_Story and @stephenb . I think my filters weren't working because I was using `rank_features` for my initial indexing. Looks like the `sparse_vector` did the trick.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 8, 2024, 1:30pm UTC](https://discuss.elastic.co/t/filter-on-search-performed-with-text-expansion-is-not-working/355110/6 "2024-04-08T13:30:11Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.