# Query to ignore fields with empty content

**URL:** <https://discuss.elastic.co/t/query-to-ignore-fields-with-empty-content/156756>\
**Category:** Elasticsearch\
**Created:** [November 14, 2018, 10:48pm UTC](https://discuss.elastic.co/t/query-to-ignore-fields-with-empty-content/156756 "2018-11-14T22:48:15Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Pradeep\_ES](https://avatars.discourse-cdn.com/v4/letter/p/3be4f8/32.png) [@Pradeep\_ES](https://discuss.elastic.co/u/Pradeep_ES)\
**Post date:** [November 14, 2018, 10:48pm UTC](https://discuss.elastic.co/t/query-to-ignore-fields-with-empty-content/156756/1 "2018-11-14T22:48:15Z")

</div>

I have the following requirements  
a) Show suggestions if a query has hits in both query\_terms and trigger\_terms  
b) Show suggestions if a query has hits in query\_terms and trigger\_terms is an empty array with no content

Here is my mapping

```
{
     "mappings": {
       "concepts": {
            "properties": {
                "query_terms": {
                    "type": "keyword"
                },
               "trigger_terms": {
                    "type": "keyword"
                },
                "suggestions": {
                    "type": "keyword"
                }
            }
        }
    }
}

```

Here is the sample data

```
Document 1:
_source
{
	"query_terms" : [
	"audit",
	"audits"
	"audited"
	],
	"trigger_terms" : [
	"audit",
	"return"
	],
	"suggestions" : '[
	"examination",
	"inspection",
	"investigation"
	]
}

Document 2:
_source
{
	"query_terms" : [
	"next",
	"test"
	"audited"
	],
	"trigger_terms" : [
	"next",
	"return",
	"test"
	],
	"suggestions" : '[
	"same offense",
	"same evidence",
	"investigation"
	]
}

Document 3:
_source
{
	"query_terms" : [
	"best",
	"fit"
	"perfect",
	"audit"
	],
	"trigger_terms" : [],
	"suggestions" : '[
	"double jeopardy",
	"same elements",
	"same evidence"
	]
}

```

And my query

```
{
	"query": {
		"bool": {
			"must": [
			    {
				  "terms": 
				  {
				    	"query_terms": ["audit"]
				  }
			   }, 
			   {
				  "terms": 
				  {
					"trigger_terms": ["audit"]
				  } 
			   }
			]
		}
	}
}

```

when I execute the query against Elasticsearch 5.2, I expect to see **Document1** ( since audit has hits in both query\_terms and trigger\_terms ) and **Document3** ( since audit has hits in query\_terms and trigger\_terms is an empty array) but I only see Document1

I modified the query but that doesn't work either

```
{
	"query": {
		"bool": {
			"must": [
			    {
				  "terms": 
				  {
				      "query_terms": ["audit"]
				  }
			   }, 
			   {
				  "terms": 
				  {
					  "trigger_terms": ["audit"]
				  } 
			   },
			   {
				  "exists" : 
				  {
			 	      "field" : "trigger_terms"
			      }
			   }
			]
		}
	}
}

```

could someone please help?

---

<div class="post-metadata">

**Author:** ![Jaspreet\_Singh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jaspreet_singh/32/58296_2.png) [@Jaspreet\_Singh](https://discuss.elastic.co/u/Jaspreet_Singh)\
**Post date:** [November 15, 2018, 6:39pm UTC](https://discuss.elastic.co/t/query-to-ignore-fields-with-empty-content/156756/2 "2018-11-15T18:39:01Z")

</div>

Is it fair to say, you want a result if there is a hit in `query_terms` but it will be nicer if there is also a hit in `trigger_terms`? Is so, you should have `query_terms` in `must` and `trigger_terms` in `should`. This way all your hits will definitely have a match in `query_terms`. Hits with a match in `trigger_terms` will be scored higher.

---

<div class="post-metadata">

**Author:** ![Pradeep\_ES](https://avatars.discourse-cdn.com/v4/letter/p/3be4f8/32.png) [@Pradeep\_ES](https://discuss.elastic.co/u/Pradeep_ES)\
**Post date:** [November 15, 2018, 8:05pm UTC](https://discuss.elastic.co/t/query-to-ignore-fields-with-empty-content/156756/3 "2018-11-15T20:05:33Z")

</div>

Thanks for replying Jaspreet! A hit in both query\_terms and trigger\_terms is a **must** in order to return suggestions and the only exception to that is when trigger\_terms is an empty array.

---

<div class="post-metadata">

**Author:** ![Jaspreet\_Singh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jaspreet_singh/32/58296_2.png) [@Jaspreet\_Singh](https://discuss.elastic.co/u/Jaspreet_Singh)\
**Post date:** [November 16, 2018, 4:12pm UTC](https://discuss.elastic.co/t/query-to-ignore-fields-with-empty-content/156756/4 "2018-11-16T16:12:47Z")

</div>

Ok, lets try this.  
Btw I added a 4th document, that has `"audit"` in `trigger_terms` but not in `query_terms`. This is another negative case, just to ensure we cover all scenarios.  
We need something like ... `(query_terms="audit" AND (trigger_terms="audit OR trigger_terms="")`  
So here is the query that worked for me ...

```auto
{
    "query": {
        "bool": {
            "must": [
                {
                    "term": {
                        "query_terms": "audit"
                    }
                },
                {
                    "bool": {
                        "should": [
                            {
                                "bool": {
                                    "must_not": [
                                        {
                                            "exists": {
                                                "field": "trigger_terms"
                                            }
                                        }
                                    ]
                                }
                            },
                            {
                                "bool": {
                                    "must": [
                                        {
                                            "term": {
                                                "trigger_terms": "audit"
                                            }
                                        }
                                    ]
                                }
                            }
                        ]
                    }
                }
            ]
        }
    }
}

```

I see the following correct results ...

```auto
{
  "took": 0,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 2,
    "max_score": 1.287682,
    "hits": [
      {
        "_index": "test1",
        "_type": "concepts",
        "_id": "3",
        "_score": 1.287682,
        "_source": {
          "query_terms": [
            "best",
            "fit",
            "perfect",
            "audit"
          ],
          "trigger_terms": [],
          "suggestions": [
            "double jeopardy",
            "same elements",
            "same evidence"
          ]
        }
      },
      {
        "_index": "test1",
        "_type": "concepts",
        "_id": "1",
        "_score": 0.5753642,
        "_source": {
          "query_terms": [
            "audit",
            "audits",
            "audited"
          ],
          "trigger_terms": [
            "audit",
            "return"
          ],
          "suggestions": [
            "examination",
            "inspection",
            "investigation"
          ]
        }
      }
    ]
  }
}

```

---

<div class="post-metadata">

**Author:** ![Pradeep\_ES](https://avatars.discourse-cdn.com/v4/letter/p/3be4f8/32.png) [@Pradeep\_ES](https://discuss.elastic.co/u/Pradeep_ES)\
**Post date:** [November 16, 2018, 8:29pm UTC](https://discuss.elastic.co/t/query-to-ignore-fields-with-empty-content/156756/5 "2018-11-16T20:29:20Z")

</div>

Thanks a lot Jaspreet! This query is working as expected.

---

<div class="post-metadata">

**Author:** ![Pradeep\_ES](https://avatars.discourse-cdn.com/v4/letter/p/3be4f8/32.png) [@Pradeep\_ES](https://discuss.elastic.co/u/Pradeep_ES)\
**Post date:** [December 3, 2018, 4:38pm UTC](https://discuss.elastic.co/t/query-to-ignore-fields-with-empty-content/156756/6 "2018-12-03T16:38:40Z")

</div>

Hi Jaspreet,

There seems to be an edge case where this is not working as expected.

Let's say we have the following document

```
{
	"query_terms" : [
	"force",	
	"audited"
	],
	"trigger_terms" : [
	"audit"	
	],
	"suggestions" : '[
	"examination",
	"inspection",
	"investigation"
	]
}

```

and my query terms are ["force", "audit"], I don't expect the above document in the response ( since force is prersent in query\_terms but not in trigger\_terms and audit is present in trigger\_terms but not in query\_terms ) but it does show up in the response.

Here is the query

```
{
    "query": {
        "bool": {
            "must": [
                {
                    "term": {
                        "query_terms": ["force", "audit"]
                    }
                },
                {
                    "bool": {
                        "should": [
                            {
                                "bool": {
                                    "must_not": [
                                        {
                                            "exists": {
                                                "field": "trigger_terms"
                                            }
                                        }
                                    ]
                                }
                            },
                            {
                                "bool": {
                                    "must": [
                                        {
                                            "term": {
                                                "trigger_terms": ["force", "audit"]
                                            }
                                        }
                                    ]
                                }
                            }
                        ]
                    }
                }
            ]
        }
    }
}

```

Could you please help?

---

<div class="post-metadata">

**Author:** ![Pradeep\_ES](https://avatars.discourse-cdn.com/v4/letter/p/3be4f8/32.png) [@Pradeep\_ES](https://discuss.elastic.co/u/Pradeep_ES)\
**Post date:** [December 11, 2018, 7:45pm UTC](https://discuss.elastic.co/t/query-to-ignore-fields-with-empty-content/156756/7 "2018-12-11T19:45:59Z")

</div>

Hi Jaspreet,

Did you get a chance to look at this?

Thanks,  
Pradeep

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 8, 2019, 7:54pm UTC](https://discuss.elastic.co/t/query-to-ignore-fields-with-empty-content/156756/8 "2019-01-08T19:54:56Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.