# Incorrect match pattern with bool/must/filter on filenames

**URL:** https://discuss.elastic.co/t/incorrect-match-pattern-with-bool-must-filter-on-filenames/88334
**Category:** Elasticsearch
**Created:** [June 5, 2017, 10:11pm UTC](https://discuss.elastic.co/t/incorrect-match-pattern-with-bool-must-filter-on-filenames/88334 "2017-06-05T22:11:18Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![Jonathan\_Delfour](https://avatars.discourse-cdn.com/v4/letter/j/bb73d2/32.png) [@Jonathan\_Delfour](https://discuss.elastic.co/u/Jonathan_Delfour)
#### Post date: [June 5, 2017, 10:11pm UTC](https://discuss.elastic.co/t/incorrect-match-pattern-with-bool-must-filter-on-filenames/88334/1 "2017-06-05T22:11:18Z")

</div>

Hi i am trying to create a simple query on file extension but somehow the search does not find all the results.

This query returns 8 hits.

```auto
     {
      "from": 0,
      "size": 10,
      "query": {
         "bool": {
                "must": [
                  {
                    "match": 
                    {
                      	"Datatype": {
                      		"type": "phrase",
                      		"query": "the type"
            	          }
        	   }
              }
           ]
         }
     }

```

response:

```auto
      {
        "_source": {
          "Datatype": "the type",   
          "FileName": "a-b.xyz",
      },
      {
        "_source": {
          "Datatype": "the type",   
          "FileName": "b-c.xyz",
      },
      {
        "_source": {
          "Datatype": "the type",   
          "FileName": "d-x.xyz",
      },
      {
        "_source": {
          "Datatype": "the type",   
          "FileName": "aa-aa.xyz",
      },
      {
        "_source": {
          "Datatype": "the type",   
          "FileName": "ddfsdf-ddf.xyz",   
      },
      {
        "_source": {
          "Datatype": "the type",   
          "FileName": "1234-sdd.xyz",
      },
      {
        "_source": {
          "Datatype": "the type",   
          "FileName": "31502-sdsd.xyz",
      },
      {
        "_source": {
            "Datatype": "the type",   
          "FileName": "16104-ss.xyz",
      }
    ]
  }
}

```

if I filter on only the `FileName`:  
I retrieve only 3 hits...

```auto
{
  "from": 0,
  "size": 10,
  "query": {
    "bool": {
      "filter": [ // tried with must/should too
      	{
      		"match": {
            	      "FileName": {
            		  "query": "xyz"
            	      }
        	}
      	}
      ]
    }
  }
}

```

I only get 3 hits on the 8 hits above while I expect to have 8. I am having a hard time to troubleshoot that.  
Any guidance on what could be going wrong?

. does not seem to be a special character. My FileName field is indexed as text and is usually of the form 123-abc.xyz

Thanks,  
Jon

---

<div class="post-metadata">

### Author: ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)
#### Post date: [June 6, 2017, 6:31am UTC](https://discuss.elastic.co/t/incorrect-match-pattern-with-bool-must-filter-on-filenames/88334/2 "2017-06-06T06:31:35Z")

</div>

Are you using the default analyzer on the FileName field? The standard  
tokenizer should decompose that field with 'xyz' as one of the terms. The  
perplexing part is that you have a match some of the time, but not all of  
the time. Seems like it would be an all-or-nothing type of scenario.

I would use the Analyze API to see exactly how the field is being analyzed  
[https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-analyze.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-analyze.html)

Use the notation where you specify the field rather than specify the  
specific analyzers and filters.

Alternatively, you can use the Explain API to see why certain documents are  
being matched:  
[https://www.elastic.co/guide/en/elasticsearch/reference/5.4/search-explain.html](https://www.elastic.co/guide/en/elasticsearch/reference/5.4/search-explain.html)  
Once you see why a certain document was matched, you can duplicate the  
logic.

---

<div class="post-metadata">

### Author: ![Jonathan\_Delfour](https://avatars.discourse-cdn.com/v4/letter/j/bb73d2/32.png) [@Jonathan\_Delfour](https://discuss.elastic.co/u/Jonathan_Delfour)
#### Post date: [June 6, 2017, 2:33pm UTC](https://discuss.elastic.co/t/incorrect-match-pattern-with-bool-must-filter-on-filenames/88334/3 "2017-06-06T14:33:44Z")

</div>

Interesting! thanks.

Here are two filenames and their analyzer response:  
`24022-ABC.xyz`

```auto
{
  "tokens": [
    {
      "token": "24022",
      "start_offset": 0,
      "end_offset": 5,
      "type": "<NUM>",
      "position": 0
    },
    {
      "token": "abc.xyz",
      "start_offset": 6,
      "end_offset": 13,
      "type": "<ALPHANUM>",
      "position": 1
    }
  ]
}

```

and `22210-ABC1.xyz`

```auto
{
  "tokens": [
    {
      "token": "22210",
      "start_offset": 0,
      "end_offset": 5,
      "type": "<NUM>",
      "position": 0
    },
    {
      "token": "abc1",
      "start_offset": 6,
      "end_offset": 10,
      "type": "<ALPHANUM>",
      "position": 1
    },
    {
      "token": "xyz",
      "start_offset": 11,
      "end_offset": 14,
      "type": "<ALPHANUM>",
      "position": 2
    }
  ]
}

```

the difference is that there is numeric character just before the . and those are all mapped.

I tried the filter part only on a index row and obtained the following, not sure how to interpret that though:

```auto
{
  "_index" : "index",
  "_type" : "ResultRow",
  "_id" : "bc593da7_994f_4b28_8075_057b2fc9cd84",
  "matched" : false,
  "explanation" : {
    "value" : 0.0,
    "description" : "Failure to meet condition(s) of required/prohibited clause(s)",
    "details" : [
      {
        "value" : 0.0,
        "description" : "no match on required clause ((ConstantScore(FileName:xyz))^0.0)",
        "details" : [
          {
            "value" : 0.0,
            "description" : "ConstantScore(FileName:xyz) doesn't match id 4697498",
            "details" : []
          }
        ]
      },
      {
        "value" : 0.0,
        "description" : "match on required clause, product of:",
        "details" : [
          {
            "value" : 0.0,
            "description" : "# clause",
            "details" : []
          },
          {
            "value" : 1.0,
            "description" : "_type:ResultRow, product of:",
            "details" : [
              {
                "value" : 1.0,
                "description" : "boost",
                "details" : []
              },
              {
                "value" : 1.0,
                "description" : "queryNorm",
                "details" : []
              }
            ]
          }
        ]
      }
    ]
  }
}

```

It seems the pattern analyzer splits things in a more appropriate way `/_analyze?text=22210-ABC.xyz&analyzer=pattern` (I get 3 tokens, 22210m abc, and xyz) but still does not work when i query:

```auto
    "filter": [
      	{
      		"match": {
            	"FileName": {
            		"analyzer":"pattern",
            		"query": "xyz"
            	}
        	}
      	}
      ]

```

---

<div class="post-metadata">

### Author: ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)
#### Post date: [June 6, 2017, 7:50pm UTC](https://discuss.elastic.co/t/incorrect-match-pattern-with-bool-must-filter-on-filenames/88334/4 "2017-06-06T19:50:49Z")

</div>

Apparently the standard tokenize does not split "ABC.xy" into separate  
tokens. The default pattern analyzer [1] should get you closer to your  
goal. If not, you can customize the pattern or use a custom analyzer to a  
pattern tokenizer.

[1]  
[https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-pattern-analyzer.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-pattern-analyzer.html)

---

<div class="post-metadata">

### Author: ![Jonathan\_Delfour](https://avatars.discourse-cdn.com/v4/letter/j/bb73d2/32.png) [@Jonathan\_Delfour](https://discuss.elastic.co/u/Jonathan_Delfour)
#### Post date: [June 6, 2017, 7:52pm UTC](https://discuss.elastic.co/t/incorrect-match-pattern-with-bool-must-filter-on-filenames/88334/5 "2017-06-06T19:52:18Z")

</div>

Yes i noticed that but somehow i am not able to apply to pattern analyzer in my query (see text block above). It just ignores the analyzer field in the json it seems.

---

<div class="post-metadata">

### Author: ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)
#### Post date: [June 7, 2017, 6:01am UTC](https://discuss.elastic.co/t/incorrect-match-pattern-with-bool-must-filter-on-filenames/88334/6 "2017-06-07T06:01:22Z")

</div>

Unfortunately, you would need to reindex your content since the correct  
tokens are not in your index. As long as you use a match query or one of  
its variants, the correct analyzer will be used. The pattern analyzer needs  
to be defined in the mapping for that field before content is indexed.

Ivan

---

<div class="post-metadata">

### Author: ![Jonathan\_Delfour](https://avatars.discourse-cdn.com/v4/letter/j/bb73d2/32.png) [@Jonathan\_Delfour](https://discuss.elastic.co/u/Jonathan_Delfour)
#### Post date: [June 19, 2017, 2:13pm UTC](https://discuss.elastic.co/t/incorrect-match-pattern-with-bool-must-filter-on-filenames/88334/7 "2017-06-19T14:13:28Z")

</div>

for those who are interested, i index the extension in a separate field.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 17, 2017, 2:13pm UTC](https://discuss.elastic.co/t/incorrect-match-pattern-with-bool-must-filter-on-filenames/88334/8 "2017-07-17T14:13:37Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.