# Aalyzer issue - terms not getting tokenized on whitespace

**URL:** https://discuss.elastic.co/t/aalyzer-issue-terms-not-getting-tokenized-on-whitespace/17072
**Category:** Elasticsearch
**Created:** [April 18, 2014, 1:41am UTC](https://discuss.elastic.co/t/aalyzer-issue-terms-not-getting-tokenized-on-whitespace/17072 "2014-04-18T01:41:25Z")
**Posts on this page:** 2
**Page:** 1

<div class="post-metadata">

### Author: ![Preeti\_Jain](https://avatars.discourse-cdn.com/v4/letter/p/67e7ee/32.png) [@Preeti\_Jain](https://discuss.elastic.co/u/Preeti_Jain)
#### Post date: [April 18, 2014, 1:41am UTC](https://discuss.elastic.co/t/aalyzer-issue-terms-not-getting-tokenized-on-whitespace/17072/1 "2014-04-18T01:41:25Z")

</div>

Hi,

We are using elasticsearch versionb 1.01.1. I did the following test for  
analyzer

PUT /test  
{  
"settings":{  
"analysis": {  
"analyzer": {  
"whitespace":{  
"type": "pattern",  
"pattern":"\\s+"  
}  
}  
}  
}  
}

GET /testallnext/\_analyze?analyzer=whitespace&pretty=1&text='Preeti,Jain  
test'

I expected that the text will be broken into 2 token Preeti,Jain and test.  
However, the result that I got was

{  
"tokens": [  
{  
"token": "'preeti,jain test'",  
"start\_offset": 0,  
"end\_offset": 18,  
"type": "word",  
"position": 1  
}  
]  
}

What is going wrong?

Our usecase requires special character to be indexed and searchable. For eg  
if I index 2 documents ,one containing text "A+B" in one of the fields and  
another one containing "A B" in the same field and then query ES for term  
"A+B", I should get only one result.  
What kind of tokenizer should be used for my purpose? I can't use  
not\_analyzed option on fields as they need to be anlyzed for phonetics and  
synonyms.

Thanks,  
Preeti

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/381fc76a-d480-4cd0-ba97-01102071518b%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/381fc76a-d480-4cd0-ba97-01102071518b%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 1:35am UTC](https://discuss.elastic.co/t/aalyzer-issue-terms-not-getting-tokenized-on-whitespace/17072/2 "2017-07-06T01:35:09Z")

</div>