# Custom Analyzer doesn't work

**URL:** <https://discuss.elastic.co/t/custom-analyzer-doesnt-work/63906>\
**Category:** Elasticsearch\
**Created:** [October 25, 2016, 4:22pm UTC](https://discuss.elastic.co/t/custom-analyzer-doesnt-work/63906 "2016-10-25T16:22:56Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jorge](https://avatars.discourse-cdn.com/v4/letter/j/9fc29f/32.png) [@Jorge](https://discuss.elastic.co/u/Jorge)\
**Post date:** [October 25, 2016, 4:22pm UTC](https://discuss.elastic.co/t/custom-analyzer-doesnt-work/63906/1 "2016-10-25T16:22:56Z")

</div>

Hellos Gays, I'm used elastic 2.3.1 with lucene 5.5.0, this is the issue, I created a custom analyzer when I test work fine, but when indexing doesn't work.

PUT test  
{

```
"analysis": {
  "analyzer": {
    "myanalyzer": {
      "type" : "custom",
      "tokenizer": "standard",
      "char_filter": ["mycharfilter"]
    }
  },
  "char_filter": {
    "mycharfilter": {
      "type": "pattern_replace",
      "pattern": "(\\d{4})(\\d{4})(\\d{4})(\\d{4})",
      "replacement": "$1$2xxx$4"
    }
  }
}

```

}

PUT /test/\_mapping/test  
{

```
"test" : {
  "properties" : {
    "texto" : {
      "type": "string",
      "analyzer": "myanalyzer"
    }
  }
}

```

}

GET test/\_mapping  
{  
"test": {  
"mappings": {  
"test": {  
"properties": {  
"texto": {  
"type": "string",  
"analyzer": "myanalyzer"  
}  
}  
}  
}  
}  
}

Look nice !! 😄

GET /test/\_analyze?analyzer=myanalyzer&text="1236852499998521"  
{  
"tokens": [  
{  
"token": "12368524xxx8521",  
"start\_offset": 1,  
"end\_offset": 17,  
"type": "",  
"position": 0  
}  
]  
}

PUT /test/test/1  
{  
"texto": "1234567812345678"  
}

Doesn't work 😭

GET /test/\_search?pretty

```
"hits": [
  {
    "_index": "test",
    "_type": "test",
    "_id": "1",
    "_score": 1,
    "_source": {
      "texto": "1234567812345678"
    }
  }
]

```

}  
}

What is wrong ?  
Thx in advance for you help

---

<div class="post-metadata">

**Author:** ![johtani](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/johtani/32/44956_2.png) [@johtani](https://discuss.elastic.co/u/johtani)\
**Post date:** [October 26, 2016, 7:38am UTC](https://discuss.elastic.co/t/custom-analyzer-doesnt-work/63906/2 "2016-10-26T07:38:27Z")

</div>

Hi @Jorge ,

`_source` is an [original JSON](https://www.elastic.co/guide/en/elasticsearch/guide/2.x/root-object.html#source-field) and does not represent analyzed string.

You can see the defined analyzer behavior by using `_analyze` API with `field` param instead of `analyzer` param.

Example :

```auto
GET /test/_analyze?field=texto&text="1236852499998521"

```

---

<div class="post-metadata">

**Author:** ![Jorge](https://avatars.discourse-cdn.com/v4/letter/j/9fc29f/32.png) [@Jorge](https://discuss.elastic.co/u/Jorge)\
**Post date:** [October 26, 2016, 3:34pm UTC](https://discuss.elastic.co/t/custom-analyzer-doesnt-work/63906/3 "2016-10-26T15:34:49Z")

</div>

Thanks for the reply. I know what you tell me. But then when the field is indexed, then it is not saved with the format analyzer?

As I can do to be indexed with the format analyzer? and when you see it displayed in the appropriate format.

Thx

---

<div class="post-metadata">

**Author:** ![johtani](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/johtani/32/44956_2.png) [@johtani](https://discuss.elastic.co/u/johtani)\
**Post date:** [October 26, 2016, 5:45pm UTC](https://discuss.elastic.co/t/custom-analyzer-doesnt-work/63906/4 "2016-10-26T17:45:33Z")

</div>

You already are indexed with the format analyzer.  
Elasticsearch does not respond indexed string data.

See : [https://www.elastic.co/guide/en/elasticsearch/guide/current/inverted-index.html](https://www.elastic.co/guide/en/elasticsearch/guide/current/inverted-index.html)

And also see: [https://www.elastic.co/guide/en/elasticsearch/guide/current/analysis-intro.html#\_when\_analyzers\_are\_used](https://www.elastic.co/guide/en/elasticsearch/guide/current/analysis-intro.html#_when_analyzers_are_used)

---

<div class="post-metadata">

**Author:** ![Jorge](https://avatars.discourse-cdn.com/v4/letter/j/9fc29f/32.png) [@Jorge](https://discuss.elastic.co/u/Jorge)\
**Post date:** [October 26, 2016, 6:23pm UTC](https://discuss.elastic.co/t/custom-analyzer-doesnt-work/63906/5 "2016-10-26T18:23:37Z")

</div>

Thanks Johtani, but I don't understand then. 2nd link says: The token is the actual term that will be stored in the index.

when I test

GET /test/\_analyze?analyzer=myanalyzer&text="1236852499998521", this is the result:

{  
"tokens": [  
{  
"token": "12368524xxx8521", \<--- this is token (Good!!!!)  
"start\_offset": 1,  
"end\_offset": 17,  
"type": "",  
"position": 0  
}  
]  
}

but if indexing

PUT /test/test/1  
{  
"texto": "1234567812345678"  
}

when I querying

GET /test/\_search?pretty

why display the field texto not formatted

"hits": [  
{  
"\_index": "test",  
"\_type": "test",  
"\_id": "1",  
"\_score": 1,  
"\_source": {  
"texto": "1234567812345678" \<-- not formatted ( 😩)  
}

Sorry if my question is fool ... but what happened ? what am I doing wrong ?

---

<div class="post-metadata">

**Author:** ![johtani](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/johtani/32/44956_2.png) [@johtani](https://discuss.elastic.co/u/johtani)\
**Post date:** [October 27, 2016, 2:09am UTC](https://discuss.elastic.co/t/custom-analyzer-doesnt-work/63906/6 "2016-10-27T02:09:00Z")

</div>

Analyzer is not formatter.  
`_source` in `_search` response is shown original JSON you indexed.

`_analyze` API show you only how analyzer tokenize `text`.  
Elasticsearch uses only each terms that tokenized by analyzer as inverted index's word.  
And Elasticsearch stores input JSON as `_source`  
But elasticsearch does not analyze `_source` data.

If you want to change original `texto` to formatted `texto` in `_source`, you should format before indexing elasticsearch or use [transform feature](https://www.elastic.co/guide/en/elasticsearch/reference/2.3/mapping-transform.html)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 10:09pm UTC](https://discuss.elastic.co/t/custom-analyzer-doesnt-work/63906/7 "2017-07-05T22:09:16Z")

</div>