Apply filter only if field exists

Hi, I'm searching across multiple index and multiple types.
But some of my types needs to be filtered by field which is not existing in all types.

my query looks like this:

curl -XGET 'http://esdb:9200/index1,index2/_search' -d '{
   "query":{
      "bool":{
         "must":[
            {
               "query_string":{
                  "query":"*komi*",
                  "rewrite":"scoring_boolean"
               }
            }
         ],
         "should":[
            {
               "term":{
                  "_type":{
                     "value":"type2",
                     "boost":5
                  }
               }
            }
         ]
      }
   },
   "size":10,
   "from":0,
   "indices_boost":{
      "index1":2,
      "index2":1
   }
}'

but to get better results I need to filter out some of the results.

so I add filter for this

"filter":[
   {
      "nested":{
         "path":"revisions",
         "query":{
            "wildcard":{
               "revisions.context":{
                  "value":"*n*",
                  "boost":1
               }
            }
         }
      }
   }
]

query:

curl -XGET 'http://esdb:9200/index1,index2/_search' -d '{
   "query":{
      "bool":{
         "must":[
            {
               "query_string":{
                  "query":"*Management*",
                  "rewrite":"scoring_boolean"
               }
            }
         ],
         "filter":[
            {
               "nested":{
                  "path":"revisions",
                  "query":{
                     "wildcard":{
                        "revisions.context":{
                           "value":"*n*",
                           "boost":1
                        }
                     }
                  }
               }
            }
         ],
         "should":[
            {
               "term":{
                  "_type":{
                     "value":"type2",
                     "boost":5
                  }
               }
            }
         ]
      }
   },
   "size":10,
   "from":0,
   "indices_boost":{
      "index1":2,
      "index2":1
   }
}'

If I apply this filter I will lost all results from types which don't have field revisions.context

It's possible to apply filter only when field exists in mappings?

Hi,

Note: you should format your code here, for readability.

For you problem, did you try the following solution ?

must ((revisions.context NOT EXISTS) OR (you revisions.context query))

bye,
Xavier

I'm to entirely sure how to make this work.
I try to add bool query with should and another bool query with must and must not. But no results from types without revisions.context are returned.

{
   "query":{
      "bool":{
         "must":[
            {
               "query_string":{
                  "query":"*Vodn\u00ed*",
                  "rewrite":"scoring_boolean"
               }
            }
         ],
         "filter":[
            {
               "bool":{
                  "should":[
                     {
                        "bool":{
                           "must_not":[
                              {
                                 "exists":{
                                    "field":"revisions.context"
                                 }
                              }
                           ],
                           "must":[
                              {
                                 "nested":{
                                    "path":"revisions",
                                    "query":{
                                       "wildcard":{
                                          "revisions.context":{
                                             "value":"*n*",
                                             "boost":1
                                          }
                                       }
                                    }
                                 }
                              }
                           ]
                        }
                     }
                  ]
               }
            }
         ],
         "should":[
            {
               "term":{
                  "_type":{
                     "value":"type2",
                     "boost":5
                  }
               }
            }
         ]
      }
   },
   "indices_boost":{
      "index1":2,
      "index2":1
   }
}

//Thank's I formated post.

you query have to be somthing like:

{
  "query":{
  "bool":{
    "must":[
	{
	  "query_string":{
	      "query":"*Vodn\u00ed*",
	      "rewrite":"scoring_boolean"
	  }
	}
    ],
    "must":[
	"bool":{
	      "minimun_should_match" : 1,
	      "should":[
		{
		  "bool":{
		      "must_not":[
			  {
			    "exists":{
				"field":"revisions.context"
			    }
			  }
		      ]
		  }
		},
		{
		    "nested":{
		      "path":"revisions",
		      "query":{
			  "wildcard":{
			    "revisions.context":{
				"value":"*n*",
				"boost":1
			    }
			  }
		      }
		    }
		}
	      ]
	  }
    ],
    "should":[
	{
	  "term":{
	      "_type":{
		"value":"type2",
		"boost":5
	      }
	  }
	}
    ]
  }
  },
  "indices_boost":{
  "index1":2,
  "index2":1
  }
}

bye,
Xavier

If I move query from filter to must and add minimum_number_should_match It will not make much difference. I get same results but this will affect scoring. Or I miss something?

{
   "query":{
      "bool":{
         "must":[
            {
               "query_string":{
                  "query":"*Vodn\u00ed*",
                  "rewrite":"scoring_boolean"
               }
            },
            {
               "query_string":{
                  "query":"*hospod\u00e1\u0159stv\u00ed*",
                  "rewrite":"scoring_boolean"
               }
            },
            {
               "bool":{
                  "minimum_number_should_match":1,
                  "should":[
                     {
                        "bool":{
                           "must_not":[
                              {
                                 "exists":{
                                    "field":"revisions.context"
                                 }
                              }
                           ],
                           "must":[
                              {
                                 "nested":{
                                    "path":"revisions",
                                    "query":{
                                       "wildcard":{
                                          "revisions.context":{
                                             "value":"*n*",
                                             "boost":1
                                          }
                                       }
                                    }
                                 }
                              }
                           ]
                        }
                     }
                  ]
               }
            }
         ],
         "should":[
            {
               "term":{
                  "_type":{
                     "value":"type2",
                     "boost":10
                  }
               }
            }
         ]
      }
   },
   "indices_boost":{
      "index1":2,
      "index2":1
   }
}

You must have 2 should clauses in your bool with minimum_number_should_match = 1
your query is:

((NOT EXISTS) OR (NESTED revisions.context="*n*")

In JSON:

"bool":{
"minimum_number_should_match":1,
"should":[
  { bool no exists },
  { nested {...}
]
}

Ii looks like that nested query is not used:

I now have query bellow. I know that for my search string there is exactly one document which match NOT EXISTS part of query. So result should be query without NOT EXISTS + 1 Document. But I get much more documents since *n* is not matched. And I get documents like revisions.context:'D', but when I just remove not exists I get right results with right context string.

//EDIT: I just try to remove only nested should part of query, and I get exact same number of results

{
   "query":{
      "bool":{
         "must":[
            {
               "query_string":{
                  ...
               }
            },
            {
               "bool":{
                  "minimum_number_should_match":1,
                  "should":[
                     {
                        "nested":{
                           "path":"revisions",
                           "query":{
                              "wildcard":{
                                 "revisions.context":{
                                    "value":"*n*"
                                 }
                              }
                           }
                        }
                     },
                     {
                        "bool":{
                           "must_not":[
                              {
                                 "exists":{
                                    "field":"revisions.context"
                                 }
                              }
                           ]
                        }
                     }
                  ]
               }
            }
         ],
         "should":[
           ...
         ]
      }
   },
   ...
}
1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.