# Search for special characters

**URL:** <https://discuss.elastic.co/t/search-for-special-characters/114495>\
**Category:** Elasticsearch\
**Created:** [January 8, 2018, 10:34am UTC](https://discuss.elastic.co/t/search-for-special-characters/114495 "2018-01-08T10:34:18Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![RohanKumbhar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rohankumbhar/32/23649_2.png) [@RohanKumbhar](https://discuss.elastic.co/u/RohanKumbhar)\
**Post date:** [January 8, 2018, 10:34am UTC](https://discuss.elastic.co/t/search-for-special-characters/114495/1 "2018-01-08T10:34:18Z")

</div>

What is the best combination of token filters+tokenizer+analyzer to search on special characters present in my document?

---

<div class="post-metadata">

**Author:** ![val](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/val/32/138203_2.png) [@val](https://discuss.elastic.co/u/val)\
**Post date:** [January 8, 2018, 1:09pm UTC](https://discuss.elastic.co/t/search-for-special-characters/114495/2 "2018-01-08T13:09:19Z")

</div>

It would be better if you provide some sample data and explain how you want to search it.

---

<div class="post-metadata">

**Author:** ![RohanKumbhar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rohankumbhar/32/23649_2.png) [@RohanKumbhar](https://discuss.elastic.co/u/RohanKumbhar)\
**Post date:** [January 8, 2018, 1:17pm UTC](https://discuss.elastic.co/t/search-for-special-characters/114495/3 "2018-01-08T13:17:31Z")

</div>

Hi @val

I have field in my document which normally contains special characters in it for example  
my\_field = "$file123#.txt"  
my\_field = "$office@location&home.txt"

now the requirement is if user search for special chars like $ , #, @, & etc. so he should get required my\_field in the result

I have tried it using ngram tokenizer but it is giving some irrelevant search result if i search for normal text ,

so now requirement here is if user search for special characters or numbers or text he should get relevant search result.

Kindly help to solve this

---

<div class="post-metadata">

**Author:** ![val](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/val/32/138203_2.png) [@val](https://discuss.elastic.co/u/val)\
**Post date:** [January 8, 2018, 1:26pm UTC](https://discuss.elastic.co/t/search-for-special-characters/114495/4 "2018-01-08T13:26:29Z")

</div>

Can you show what you tried with ngrams?

---

<div class="post-metadata">

**Author:** ![RohanKumbhar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rohankumbhar/32/23649_2.png) [@RohanKumbhar](https://discuss.elastic.co/u/RohanKumbhar)\
**Post date:** [January 8, 2018, 1:44pm UTC](https://discuss.elastic.co/t/search-for-special-characters/114495/5 "2018-01-08T13:44:29Z")

</div>

I have used below custom analyzer

```
PUT my_index
{
  "settings": {
    "index": {
      "analysis": {
        "analyzer": {
          "custom_analyzer": {
            "type": "custom",
            "tokenizer": "my_tokenizer"
          }
        },
        "tokenizer": {
          "my_tokenizer": {
            "type": "ngram",
            "min_gram": 1,
            "max_gram": 10
          }
        }
      }
    }
  },
  "mappings": {
    "doc": {
      "properties": {
        "my_field": {
          "type": "text",
          "analyzer": "custom_analyzer",
          "fields": {
            "keyword": {
              "ignore_above": 256,
              "type": "keyword"
            }
          }
        }
      }
    }
  }
}

PUT my_index/doc/2 
{
  "my_field":"$title123.txt"
}

PUT my_index/doc/1
{
  "my_field":"$titan@123#.txt"
}

```

Here special character search is working fine but if i search for **titan** i'm getting both the documents (my\_field with **title** in it) which is irrelevant search

---

<div class="post-metadata">

**Author:** ![val](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/val/32/138203_2.png) [@val](https://discuss.elastic.co/u/val)\
**Post date:** [January 8, 2018, 1:49pm UTC](https://discuss.elastic.co/t/search-for-special-characters/114495/6 "2018-01-08T13:49:15Z")

</div>

Great start! Try not to use your custom analyzer at search time, otherwise the search terms will get analyzed as well, i.e. `titan` will get tokenized to `t`, `ti`, `tit`, `tita` and `titan` and of course the first three tokens will match `title` too.

Add `"search_analyzer" => "standard"` to your field and it should already work better

```
    "my_field": {
      "type": "text",
      "analyzer": "custom_analyzer",
      "search_analyzer": "standard", <--- add this
      "fields": {
        "keyword": {
          "ignore_above": 256,
          "type": "keyword"
        }
      }
    }

```

Note that you need to re-create and re-index your data first.

---

<div class="post-metadata">

**Author:** ![val](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/val/32/138203_2.png) [@val](https://discuss.elastic.co/u/val)\
**Post date:** [January 8, 2018, 1:54pm UTC](https://discuss.elastic.co/t/search-for-special-characters/114495/7 "2018-01-08T13:54:06Z")

</div>

Also note that standard analyzer I'm suggesting might not be a good fit for searching special characters. You might want to create another custom analyzer but without the ngram tokenizer. **The key point is not to use ngrams at search time.**

---

<div class="post-metadata">

**Author:** ![RohanKumbhar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rohankumbhar/32/23649_2.png) [@RohanKumbhar](https://discuss.elastic.co/u/RohanKumbhar)\
**Post date:** [January 8, 2018, 1:55pm UTC](https://discuss.elastic.co/t/search-for-special-characters/114495/8 "2018-01-08T13:55:25Z")

</div>

Sure , Thanks @val !!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 5, 2018, 2:08pm UTC](https://discuss.elastic.co/t/search-for-special-characters/114495/9 "2018-02-05T14:08:08Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.