# Trouble finding the documents I want (whitespace analyzer)

**URL:** <https://discuss.elastic.co/t/trouble-finding-the-documents-i-want-whitespace-analyzer/9072>\
**Category:** Elasticsearch\
**Created:** [September 20, 2012, 9:44am UTC](https://discuss.elastic.co/t/trouble-finding-the-documents-i-want-whitespace-analyzer/9072 "2012-09-20T09:44:02Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![SomeUser](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/someuser/32/2423_2.png) [@SomeUser](https://discuss.elastic.co/u/SomeUser)\
**Post date:** [September 20, 2012, 9:44am UTC](https://discuss.elastic.co/t/trouble-finding-the-documents-i-want-whitespace-analyzer/9072/1 "2012-09-20T09:44:02Z")

</div>

I'm having some trouble setting up an index and getting my queries to find  
the documents I want.

I have a field containing human written text and can contain emails, URLs,  
FQDNs, IP addresses, names with "-" in them e.g. Mary-Anne etc. When I for  
instance search for an email ([abc.def@qwe.com](mailto:abc.def@qwe.com)\*) I get matches for documents  
containing "abc", "def", "qwe", "com" which is a problem since many users  
have an email from the same domain (such as @gmail.com).

If I write "ab\*" I want to find documents containing "abc.def@qwe.com" or  
"abort" in the same way as writing "addr\*" finds documents containing  
"address". If I write "abc.def@qwe.com" I want to find documents containing  
"abc.def@qwe.com" and not for example "xyz@qwe.com".

What am I missing? How should I index and query for text containing these  
types of tokens?

Mapping:  
{  
"mappings": {  
"mytype": {  
"properties": {  
"email": {  
"type": "string",  
"analyzer": "uax\_url\_email"  
},  
"status": {  
"type": "string",  
"index": "not\_analyzed"  
},  
"description": {  
"type": "string",  
"analyzer": "custom\_whitespace"  
}  
}  
}  
},  
"settings": {  
"analysis": {  
"analyzer": {  
"uax\_url\_email": {  
"filter": ["standard",  
"lowercase",  
"stop"],  
"tokenizer": "uax\_url\_email"  
},  
"custom\_whitespace": {  
"filter": ["lowercase"],  
"tokenizer": "whitespace"  
}  
}  
}  
}  
}

Sample document  
{  
status: "somestatus",  
email: "abc.def@qwe.com",  
Description: "Description about something containing some difficult to  
search for terms 192.168.0.1, [abc.def@qwe.com](mailto:abc.def@qwe.com), Mary-Ann etc"  
}

Query  
{  
"query": {  
"query\_string": {  
"query": "abc.def@qwe.com\*",  
"default\_operator": "AND",  
"analyze\_wildcard": true  
}  
}  
}

--

---

<div class="post-metadata">

**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\
**Post date:** [September 20, 2012, 5:35pm UTC](https://discuss.elastic.co/t/trouble-finding-the-documents-i-want-whitespace-analyzer/9072/2 "2012-09-20T17:35:43Z")

</div>

When using a query\_string query, you need to specific default\_field.  
If not, the \_all field is used, which uses the standard analyzer.

--  
Ivan

On Thu, Sep 20, 2012 at 2:44 AM, SomeUser [fannkeentoong@gmail.com](mailto:fannkeentoong@gmail.com) wrote:

> I'm having some trouble setting up an index and getting my queries to find  
> the documents I want.
> 
> I have a field containing human written text and can contain emails, URLs,  
> FQDNs, IP addresses, names with "-" in them e.g. Mary-Anne etc. When I for  
> instance search for an email ([abc.def@qwe.com](mailto:abc.def@qwe.com)\*) I get matches for documents  
> containing "abc", "def", "qwe", "com" which is a problem since many users  
> have an email from the same domain (such as @gmail.com).
> 
> If I write "ab\*" I want to find documents containing "[abc.def@qwe.com](mailto:abc.def@qwe.com)" or  
> "abort" in the same way as writing "addr\*" finds documents containing  
> "address". If I write "[abc.def@qwe.com](mailto:abc.def@qwe.com)" I want to find documents containing  
> "[abc.def@qwe.com](mailto:abc.def@qwe.com)" and not for example "[xyz@qwe.com](mailto:xyz@qwe.com)".
> 
> What am I missing? How should I index and query for text containing these  
> types of tokens?
> 
> Mapping:  
> {  
> "mappings": {  
> "mytype": {  
> "properties": {  
> "email": {  
> "type": "string",  
> "analyzer": "uax\_url\_email"  
> },  
> "status": {  
> "type": "string",  
> "index": "not\_analyzed"  
> },  
> "description": {  
> "type": "string",  
> "analyzer": "custom\_whitespace"  
> }  
> }  
> }  
> },  
> "settings": {  
> "analysis": {  
> "analyzer": {  
> "uax\_url\_email": {  
> "filter": ["standard",  
> "lowercase",  
> "stop"],  
> "tokenizer": "uax\_url\_email"  
> },  
> "custom\_whitespace": {  
> "filter": ["lowercase"],  
> "tokenizer": "whitespace"  
> }  
> }  
> }  
> }  
> }
> 
> Sample document  
> {  
> status: "somestatus",  
> email: "[abc.def@qwe.com](mailto:abc.def@qwe.com)",  
> Description: "Description about something containing some difficult to  
> search for terms 192.168.0.1, [abc.def@qwe.com](mailto:abc.def@qwe.com), Mary-Ann etc"  
> }
> 
> Query  
> {  
> "query": {  
> "query\_string": {  
> "query": "[abc.def@qwe.com](mailto:abc.def@qwe.com)\*",  
> "default\_operator": "AND",  
> "analyze\_wildcard": true  
> }  
> }  
> }
> 
> --

--

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 3:12am UTC](https://discuss.elastic.co/t/trouble-finding-the-documents-i-want-whitespace-analyzer/9072/3 "2017-07-06T03:12:07Z")

</div>