# Tokens not shown in analysis API is searchable

**URL:** <https://discuss.elastic.co/t/tokens-not-shown-in-analysis-api-is-searchable/8666>\
**Category:** Elasticsearch\
**Created:** [August 8, 2012, 8:37am UTC](https://discuss.elastic.co/t/tokens-not-shown-in-analysis-api-is-searchable/8666 "2012-08-08T08:37:24Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![vineeth\_mohan](https://avatars.discourse-cdn.com/v4/letter/v/bc79bd/32.png) [@vineeth\_mohan](https://discuss.elastic.co/u/vineeth_mohan)\
**Post date:** [August 8, 2012, 8:37am UTC](https://discuss.elastic.co/t/tokens-not-shown-in-analysis-api-is-searchable/8666/1 "2012-08-08T08:37:24Z")

</div>

Hi ,

I have a index where i have applied html\_strip character filter with  
lowercase , stopword and snowball filter.  
But then am able to search words like href or some words in the link  
provided in the href.

The strange case is that , when i run the same feed against the analysis  
API , am not seeing these tokens.  
Content is the field where i have applied the aforementioned analyzer.

FEED - [https://gist.github.com/3293411](https://gist.github.com/3293411)  
on hitting ANALYSIS API - [https://gist.github.com/3293432](https://gist.github.com/3293432)  
Command used to create analyzer and index - [https://gist.github.com/3293443](https://gist.github.com/3293443)  
Output of TermList plugin - [https://gist.github.com/3293448](https://gist.github.com/3293448)

Here list given by analysis API is different from termlist plugin.

In short html\_strip is working while using the analysis API but not working  
when actual indexing happens.

Thanks  
Vineeth

---

<div class="post-metadata">

**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\
**Post date:** [August 8, 2012, 12:48pm UTC](https://discuss.elastic.co/t/tokens-not-shown-in-analysis-api-is-searchable/8666/2 "2012-08-08T12:48:55Z")

</div>

These tokens are coming from the \_all field  
([Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/mapping/all-field.html)) which  
is indexed by default analyzer by default. As far as I can see from your  
mapping, you are not overriding the default analyzer.

On Wednesday, August 8, 2012 4:37:24 AM UTC-4, Vineeth Mohan wrote:

> Hi ,
> 
> I have a index where i have applied html\_strip character filter with  
> lowercase , stopword and snowball filter.  
> But then am able to search words like href or some words in the link  
> provided in the href.
> 
> The strange case is that , when i run the same feed against the analysis  
> API , am not seeing these tokens.  
> Content is the field where i have applied the aforementioned analyzer.
> 
> FEED - [gist:3293411 · GitHub](https://gist.github.com/3293411)  
> on hitting ANALYSIS API - [gist:3293432 · GitHub](https://gist.github.com/3293432)  
> Command used to create analyzer and index -  
> [gist:3293443 · GitHub](https://gist.github.com/3293443)  
> Output of TermList plugin - [gist:3293448 · GitHub](https://gist.github.com/3293448)
> 
> Here list given by analysis API is different from termlist plugin.
> 
> In short html\_strip is working while using the analysis API but not  
> working when actual indexing happens.
> 
> Thanks  
> Vineeth

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 3:17am UTC](https://discuss.elastic.co/t/tokens-not-shown-in-analysis-api-is-searchable/8666/3 "2017-07-06T03:17:13Z")

</div>