# Synonyms are always whitespace tokenized?

**URL:** <https://discuss.elastic.co/t/synonyms-are-always-whitespace-tokenized/13780>\
**Category:** Elasticsearch\
**Created:** [September 27, 2013, 12:05am UTC](https://discuss.elastic.co/t/synonyms-are-always-whitespace-tokenized/13780 "2013-09-27T00:05:55Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Glen\_Smith](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/glen_smith/32/111656_2.png) [@Glen\_Smith](https://discuss.elastic.co/u/Glen_Smith)\
**Post date:** [September 27, 2013, 12:05am UTC](https://discuss.elastic.co/t/synonyms-are-always-whitespace-tokenized/13780/1 "2013-09-27T00:05:55Z")

</div>

I am trying to use a synonym list that includes space-separated values.  
For example, if a "1" is passed in a match query to the configured field, I  
want to expand that into a stream of tokens that  
includes "Phase 1". However, as far as I can tell, entries in the synonym  
list get tokenized by whitespace.

Is this _really_ the only behavior available?

Running this:  
#!/bin/sh  
echo "\nattempt to delete the index"  
curl -XDELETE "[http://localhost:9200/syndex/?pretty=false](http://localhost:9200/syndex/?pretty=false)"  
echo "\ncreate the index"  
curl -XPUT "[http://localhost:9200/syndex/?pretty=true](http://localhost:9200/syndex/?pretty=true)" -d '{  
"settings": {  
"number\_of\_shards": 1,  
"number\_of\_replicas": 0,  
"analysis": {  
"analyzer": {  
"syn": {  
"tokenizer": "keyword",  
"filter": ["color\_synonym"]  
}  
},  
"filter": {  
"color\_synonym": {  
"type" : "synonym",  
"synonyms" : ["red, another shade"]  
}  
}  
}  
}  
}'  
echo "\n analyze red: I want a single token _another shade_"  
echo "\n instead I get two tokens _another_ & _shade_"  
curl -XGET "localhost:9200/syndex/\_analyze?analyzer=syn&pretty=true" -d  
"red"  
echo "\n sanity check how keyword tokenizer handes another shade"  
curl -XGET "localhost:9200/syndex/\_analyze?analyzer=syn&pretty=true" -d  
"another shade"  
echo "\n and of course it does not split them"

Generates the output:  
attempt to delete the index  
{"ok":true,"acknowledged":true}  
create the index  
{  
"ok" : true,  
"acknowledged" : true  
}  
analyze red: I want a single token _another shade_

instead I get two tokens _another_ & _shade_  
{  
"tokens" : [ {  
"token" : "red",  
"start\_offset" : 0,  
"end\_offset" : 3,  
"type" : "SYNONYM",  
"position" : 1  
}, {  
"token" : "another",  
"start\_offset" : 0,  
"end\_offset" : 3,  
"type" : "SYNONYM",  
"position" : 1  
}, {  
"token" : "shade",  
"start\_offset" : 0,  
"end\_offset" : 3,  
"type" : "SYNONYM",  
"position" : 2  
} ]  
}  
sanity check how keyword tokenizer handes another shade  
{  
"tokens" : [ {  
"token" : "another shade",  
"start\_offset" : 0,  
"end\_offset" : 13,  
"type" : "word",  
"position" : 1  
} ]  
}  
and of course it does not split them

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 2:14am UTC](https://discuss.elastic.co/t/synonyms-are-always-whitespace-tokenized/13780/2 "2017-07-06T02:14:28Z")

</div>