# ES doesn't remove Persian stopwords

**URL:** <https://discuss.elastic.co/t/es-doesnt-remove-persian-stopwords/92949>\
**Category:** Elasticsearch\
**Created:** [July 13, 2017, 7:30am UTC](https://discuss.elastic.co/t/es-doesnt-remove-persian-stopwords/92949 "2017-07-13T07:30:15Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ahmad\_Ahmadi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ahmad_ahmadi/32/13590_2.png) [@Ahmad\_Ahmadi](https://discuss.elastic.co/u/Ahmad_Ahmadi)\
**Post date:** [July 13, 2017, 7:30am UTC](https://discuss.elastic.co/t/es-doesnt-remove-persian-stopwords/92949/1 "2017-07-13T07:30:15Z")

</div>

I made a custom analyzer based on Persian analyzer to use my custom stopwords. The problem is ES doesn't remove Persian words from the text.

The analyzer:  
persian-without-stopwords-analyzer: {  
type: "persian",  
stopwords: [  
"something",  
"دبیرستان",  
"another"  
]  
}

And I tested my analyzer using the following code:  
GET driq/\_analyze  
{  
"analyzer": "persian-without-stopwords-analyzer",  
"text" : "something دبیرستان another"  
}

The result is:  
{  
"tokens": [  
{  
"token": "دبيرستان",  
"start\_offset": 10,  
"end\_offset": 18,  
"type": "",  
"position": 1  
}  
]  
}

Why it removes 'something' and 'another' but not 'دبیرستان'?

---

<div class="post-metadata">

**Author:** ![Ahmad\_Ahmadi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ahmad_ahmadi/32/13590_2.png) [@Ahmad\_Ahmadi](https://discuss.elastic.co/u/Ahmad_Ahmadi)\
**Post date:** [July 13, 2017, 8:30am UTC](https://discuss.elastic.co/t/es-doesnt-remove-persian-stopwords/92949/2 "2017-07-13T08:30:10Z")

</div>

I found the solution. Because of normalization filters Persian 'ye' character was replaced with Arabic 'ye', but I write the stopword with Persian 'ye', so the didn't match. I write my stopword using Arabic 'ye' and ES removes stopwords from index.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 10, 2017, 8:30am UTC](https://discuss.elastic.co/t/es-doesnt-remove-persian-stopwords/92949/3 "2017-08-10T08:30:41Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.