\
**Category:** Elasticsearch\
**Created:** [August 3, 2012, 5:51am UTC](https://discuss.elastic.co/t/strange-behaviour-wtih-html-strip-and-stemmer-filter-combination/8622 "2012-08-03T05:51:27Z")\
**Posts on this page:** 4\
**Page:** 1
**Author:**  [@vineeth\_mohan](https://discuss.elastic.co/u/vineeth_mohan)\
**Post date:** [August 3, 2012, 5:51am UTC](https://discuss.elastic.co/t/strange-behaviour-wtih-html-strip-and-stemmer-filter-combination/8622/1 "2012-08-03T05:51:27Z")
Hi ,
I am using this script to create a custom analyzer which applies a stemmer
and html\_strip.
SCRIPT - [https://gist.github.com/3244856](https://gist.github.com/3244856)
But in the result , the word "thi" is recognized as a token (This is
somehow obtained from the word this)
Output - [https://gist.github.com/3244869](https://gist.github.com/3244869)
While not using html\_strip , this is not happening. It remove the token
this as expected.
Thanks
Vineeth
---
**Author:**  [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\
**Post date:** [August 4, 2012, 1:23am UTC](https://discuss.elastic.co/t/strange-behaviour-wtih-html-strip-and-stemmer-filter-combination/8622/2 "2012-08-04T01:23:40Z")
Vineeth,
In case of the "content" analyzer, your text goes through the stemmer
filter before it reaches the stop word filter. When the token "this" goes
through the stemmer filter it is converted into "thi". The token "thi" is
not a stop word and therefore the stop word filter passes it through. It
might make sense to put the stop word filter before the stemmer filter to
avoid this problem:
```
"filter" : ["lowercase", "stop", "stemmer"],
```
Igor
On Friday, August 3, 2012 1:51:27 AM UTC-4, Vineeth Mohan wrote:
> Hi ,
>
> I am using this script to create a custom analyzer which applies a stemmer
> and html\_strip.
>
> SCRIPT - [gist:3244856 · GitHub](https://gist.github.com/3244856)
>
> But in the result , the word "thi" is recognized as a token (This is
> somehow obtained from the word this)
>
> Output - [gist:3244869 · GitHub](https://gist.github.com/3244869)
>
> While not using html\_strip , this is not happening. It remove the token
> this as expected.
>
> Thanks
> Vineeth
---
**Author:**  [@vineeth\_mohan](https://discuss.elastic.co/u/vineeth_mohan)\
**Post date:** [August 4, 2012, 1:54am UTC](https://discuss.elastic.co/t/strange-behaviour-wtih-html-strip-and-stemmer-filter-combination/8622/3 "2012-08-04T01:54:15Z")
Hi ,
From our chat , it turns out the default stemmer is porter stemmer and it
does this operation as i havnt mentioned the language.
Thanks
Vineeth
On Sat, Aug 4, 2012 at 6:53 AM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com) wrote:
> Vineeth,
>
> In case of the "content" analyzer, your text goes through the stemmer
> filter before it reaches the stop word filter. When the token "this" goes
> through the stemmer filter it is converted into "thi". The token "thi" is
> not a stop word and therefore the stop word filter passes it through. It
> might make sense to put the stop word filter before the stemmer filter to
> avoid this problem:
>
> ```
> "filter" : ["lowercase", "stop", "stemmer"],
>
> ```
>
> Igor
>
> On Friday, August 3, 2012 1:51:27 AM UTC-4, Vineeth Mohan wrote:
>
> > Hi ,
> >
> > I am using this script to create a custom analyzer which applies a
> > stemmer and html\_strip.
> >
> > SCRIPT - [https://gist.github.com/\*\*3244856](https://gist.github.com/**3244856)[https://gist.github.com/3244856](https://gist.github.com/3244856)
> >
> > But in the result , the word "thi" is recognized as a token (This is
> > somehow obtained from the word this)
> >
> > Output - [https://gist.github.com/\*\*3244869](https://gist.github.com/**3244869)[https://gist.github.com/3244869](https://gist.github.com/3244869)
> >
> > While not using html\_strip , this is not happening. It remove the token
> > this as expected.
> >
> > Thanks
> > Vineeth
---
**Author:**  [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 3:17am UTC](https://discuss.elastic.co/t/strange-behaviour-wtih-html-strip-and-stemmer-filter-combination/8622/4 "2017-07-06T03:17:44Z")
Closed on July 6, 2017, 3:17am UTC