# How to Discard Empty Fields When Searching?

**URL:** <https://discuss.elastic.co/t/how-to-discard-empty-fields-when-searching/14009>\
**Category:** Elasticsearch\
**Created:** [October 18, 2013, 10:22am UTC](https://discuss.elastic.co/t/how-to-discard-empty-fields-when-searching/14009 "2013-10-18T10:22:16Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Eric\_4](https://avatars.discourse-cdn.com/v4/letter/e/f1d935/32.png) [@Eric\_4](https://discuss.elastic.co/u/Eric_4)\
**Post date:** [October 18, 2013, 10:22am UTC](https://discuss.elastic.co/t/how-to-discard-empty-fields-when-searching/14009/1 "2013-10-18T10:22:16Z")

</div>

Hi,

I am running following query. Basically I want to exclude all the documents  
where _lastname_ starts with a number. The first set of documents in the  
result set are the ones _where lastname field is empty_. I simply want to  
discard the documents where the lastname field is empty. How do I do that?  
I tried "missing" : "\_last" but as I expected it seems to work only when  
the field is missing from the document. I am using a custom PHP class to  
generate the query and therefore I have to use the "query\_string" query.

Thank you.

{  
"filter": {  
"and": [  
{  
"query": {  
"query\_string": {  
"query": "active",  
"default\_field": "status"  
}  
}  
},  
{  
"query": {  
"query\_string": {  
"query": "+lastname:(-0\* -1\* -2\* -3\* -4\* -5\* -6\* -7\* -8\* -9\*)",  
"default\_field": "lastname"  
}  
}  
}  
]  
},  
"sort": [  
{  
"lastname\_not\_analyzed": {  
"order": "asc",  
"missing" : "\_last"  
}  
}  
],  
"size": 10,  
"from": 0  
}

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\
**Post date:** [October 18, 2013, 5:16pm UTC](https://discuss.elastic.co/t/how-to-discard-empty-fields-when-searching/14009/2 "2013-10-18T17:16:31Z")

</div>

Easiest thing would be just to exclude empty lastnames, or lastnames  
beginning with a number:

POST /\_search  
{  
"query": {  
"filtered": {  
"filter": {  
"bool": {  
"must": [  
{  
"term": {  
"status": "active"  
}  
},  
{  
"exists": {  
"field": "lastname"  
}  
}  
],  
"must\_not": [  
{  
"regexp": {  
"lastname": "[0-9].+"  
}  
}  
]  
}  
}  
}  
},  
"sort": [  
{  
"lastname\_not\_analyzed": {  
"order": "asc"  
}  
}  
]  
}

Notes:

1. instead of having a separate field called lastname\_not\_analyzed, look at  
using multi\_fields

[http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-multi-field-type.html](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-multi-field-type.html)

1. running the regexp against the lastname will match against any word in  
the lastname, not just the first.  
perhaps you want to match against lastname\_not\_analyzed instead?

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Eric\_4](https://avatars.discourse-cdn.com/v4/letter/e/f1d935/32.png) [@Eric\_4](https://discuss.elastic.co/u/Eric_4)\
**Post date:** [October 20, 2013, 2:43pm UTC](https://discuss.elastic.co/t/how-to-discard-empty-fields-when-searching/14009/3 "2013-10-20T14:43:46Z")

</div>

Hi Clinton,

Thanks for the answer. I will try the suggestions.

Is there any particular reason for this behavior (including and showing the  
documents with empty fields first).

When it comes to sorting, why does it lists documents with empty fields  
first (In this case, when we sort based on lastname, it lists the documents  
where lastname is empty first)?

Thank you.

On Friday, October 18, 2013 6:22:16 PM UTC+8, Eric wrote:

> Hi,
> 
> I am running following query. Basically I want to exclude all the  
> documents where _lastname_ starts with a number. The first set of  
> documents in the result set are the ones _where lastname field is empty_.  
> I simply want to discard the documents where the lastname field is empty.  
> How do I do that? I tried "missing" : "\_last" but as I expected it seems to  
> work only when the field is missing from the document. I am using a custom  
> PHP class to generate the query and therefore I have to use  
> the "query\_string" query.
> 
> Thank you.
> 
> {  
> "filter": {  
> "and": [  
> {  
> "query": {  
> "query\_string": {  
> "query": "active",  
> "default\_field": "status"  
> }  
> }  
> },  
> {  
> "query": {  
> "query\_string": {  
> "query": "+lastname:(-0\* -1\* -2\* -3\* -4\* -5\* -6\* -7\* -8\* -9\*)",  
> "default\_field": "lastname"  
> }  
> }  
> }  
> ]  
> },  
> "sort": [  
> {  
> "lastname\_not\_analyzed": {  
> "order": "asc",  
> "missing" : "\_last"  
> }  
> }  
> ],  
> "size": 10,  
> "from": 0  
> }

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 2:11am UTC](https://discuss.elastic.co/t/how-to-discard-empty-fields-when-searching/14009/4 "2017-07-06T02:11:36Z")

</div>


