# Question on \_analyze handler defaults

**URL:** <https://discuss.elastic.co/t/question-on--analyze-handler-defaults/28813>\
**Category:** Elasticsearch\
**Created:** [September 8, 2015, 6:56am UTC](https://discuss.elastic.co/t/question-on--analyze-handler-defaults/28813 "2015-09-08T06:56:51Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![mdomans](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mdomans/32/4625_2.png) [@mdomans](https://discuss.elastic.co/u/mdomans)\
**Post date:** [September 8, 2015, 6:56am UTC](https://discuss.elastic.co/t/question-on--analyze-handler-defaults/28813/1 "2015-09-08T06:56:51Z")

</div>

Something that boggles me in regard to how \_analyze actually works:

Here's an example of request I'm running

```auto
GET /_analyze?char_filter=html_strip
{"I really can swim with my gill at the CafÃ! "}

```

What puzzles me in the output is the lack of **!** character in the result due to the fact **!** not being a html character. What's more, I'm actually seeing tokens there, which suggests there's a tokenizer being used.

My question: what are the default filters and tokenizer for \_analyze handler and how can I turn them off?

Full output here:

```auto
{
   "tokens": [
      {
         "token": "i",
         "start_offset": 2,
         "end_offset": 3,
         "type": "<ALPHANUM>",
         "position": 1
      },
      {
         "token": "really",
         "start_offset": 4,
         "end_offset": 10,
         "type": "<ALPHANUM>",
         "position": 2
      },
      {
         "token": "can",
         "start_offset": 11,
         "end_offset": 14,
         "type": "<ALPHANUM>",
         "position": 3
      },
      {
         "token": "swim",
         "start_offset": 15,
         "end_offset": 19,
         "type": "<ALPHANUM>",
         "position": 4
      },
      {
         "token": "with",
         "start_offset": 20,
         "end_offset": 24,
         "type": "<ALPHANUM>",
         "position": 5
      },
      {
         "token": "my",
         "start_offset": 25,
         "end_offset": 27,
         "type": "<ALPHANUM>",
         "position": 6
      },
      {
         "token": "gill",
         "start_offset": 28,
         "end_offset": 32,
         "type": "<ALPHANUM>",
         "position": 7
      },
      {
         "token": "at",
         "start_offset": 33,
         "end_offset": 35,
         "type": "<ALPHANUM>",
         "position": 8
      },
      {
         "token": "the",
         "start_offset": 36,
         "end_offset": 39,
         "type": "<ALPHANUM>",
         "position": 9
      },
      {
         "token": "cafã",
         "start_offset": 40,
         "end_offset": 44,
         "type": "<ALPHANUM>",
         "position": 10
      }
   ]
}

```

---

<div class="post-metadata">

**Author:** ![colings86](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/colings86/32/44960_2.png) [@colings86](https://discuss.elastic.co/u/colings86)\
**Post date:** [September 8, 2015, 7:08am UTC](https://discuss.elastic.co/t/question-on--analyze-handler-defaults/28813/2 "2015-09-08T07:08:07Z")

</div>

You cannot use a token filter without a tokenizer. The input to a token filter is a stream of tokens rather than a string so you need the tokenizer to create this stream of tokens to pass to your token filter. The default tokenizer is the standard tokenizer which splits token on whitespace and other punctuation and removes the whitespace and punctuation. This is why you are seeing the `!` disappear in your example above.

If you want to test the `html_strip` filter in isolation I would suggest adding `tokenizer=keyword` to your URL query parameters. The keyword tokenizer takes the input string and outputs a single token containing the full unmodified text of the input string. For other tokenizers which you might want to use instead see here: [https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-tokenizers.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-tokenizers.html)

Hope that helps

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [September 8, 2015, 7:09am UTC](https://discuss.elastic.co/t/question-on--analyze-handler-defaults/28813/3 "2015-09-08T07:09:29Z")

</div>

By default it'll use the [standard analyzer](https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-standard-analyzer.html) but you can override that by explicitly setting another analyzer (perhaps [whitespace](https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-whitespace-analyzer.html) is what you're looking for?) and/or overriding which tokenizers and filters to use.

---

<div class="post-metadata">

**Author:** ![mdomans](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mdomans/32/4625_2.png) [@mdomans](https://discuss.elastic.co/u/mdomans)\
**Post date:** [September 8, 2015, 7:23am UTC](https://discuss.elastic.co/t/question-on--analyze-handler-defaults/28813/4 "2015-09-08T07:23:53Z")

</div>

@colings86 thank you - that's what I was looking for. And just in case somebody is doing already something similar - I'm writing a simple piece of code to run text against permutations of a set of filters/char\_filters/tokenizers to help me with writing custom analyzers.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 11:51pm UTC](https://discuss.elastic.co/t/question-on--analyze-handler-defaults/28813/5 "2017-07-05T23:51:44Z")

</div>