Exclude searching on specific fields

Elastic Stack Elasticsearch
#1

Hello,

Is there any way to NOT search on a specific field while performing a full text based search using the Query DSL / Query String Query syntax, following is the index i am creating.

{
"e3c95a54-2971-47ec-b177-1fdbe9e04639": {
"mappings": {
"81f71ece-9acc-4750-82f6-53fe47bd4489": {
"properties": {
"docfilename": {
"type": "string"
},
"usercomments": {
"type": "string"
},
"filepath": {
"type": "string",
"fields": {
"raw": {
"type": "string",
"index": "not_analyzed"
}
}
},
"singlewordfield": {
"type": "string",
"fields": {
"raw": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}
}
}
}

{
"docfilename": "Profile.doc",
"usercomments": "this is a sample comment entered by the user.",
"filepath": "\path\Profile.doc",
"singlewordfield": "apple"
}

using the following query string syntax i want to search on all the fields except the usercomments field, is this possible using the query string syntax, or is there any other approach to acheive this.

{"query" :{"query_string":{"query":"profile"}}}

Thanks,
Vikas.

#2

Hi Vikas,

By default, the query-string-query searches within the "_all field" field. This is a field which contains all values of your document concatenated into one big string.
If you want to remove one of your fields from the _all field, you can do so via your mapping with the 'include_in_all' setting, as described here.

Your mapping would look like the following:

{
  "mappings": {
    "81f71ece-9acc-4750-82f6-53fe47bd4489": {
      "properties": {
        "docfilename": {
          "type": "string"
        },
        "usercomments": {
          "type": "string",
          "include_in_all" : false
        },
        "filepath": {
          "type": "string",
          "fields": {
            "raw": {
              "type": "string",
              "index": "not_analyzed"
            }
          }
        },
        "singlewordfield": {
          "type": "string",
          "fields": {
            "raw": {
              "type": "string",
              "index": "not_analyzed"
            }
          }
        }
      }
    }
  }
}

Since your 'usercomments' data won't exist in the '_all field', you will not match against data from that field anymore.
One other solution is to replace your query-string-query with either a multi-match-query or a combination of bool and match queries.

Hope this helps :slight_smile:

2 Likes
#3

Thanks for the superquick response,

I have an additional query on this, how do we go about setting the "include_in_all" mapping for a field which would be added in the index at a later stage, since the field name would be decided at that point in time & hence cannot include the future (new) field in the mapping request upfront.

Agree to your alternate solutions of using the multi-match / math queries but I have a constraint of using query-string-query due to the ease of sending search requests, as the requests are floated from a UI based app (the users type in the query-string-query syntax on a UI provided).

Thanks again,
Vikas.

#4

Hi Vikas,

If you know up front which fields you want to query on, you could add the 'fields' parameter to your query string query. The default field is the '_all' field, but you can also specify a list of fields to search through:

GET /_search
{
  "query": {
    "query_string": {
      "query": "this AND that OR thus",
      "fields": [
          "docfilename",
          "filepath",
          "singlewordfield"
        ]
    }
  }
}

If that doesn't meet your requirement, there might be a way of identifying prefixes in the fields to be made. For example, if all new fields start with 'foo' (like foo132, foo2bar etc..), you could create a 'default index template' where you can use wildcards to those fields to be not included:

 "dynamic_templates": [
                {
                    "string_template" : { 
                        "match" : "foo*",
                        "mapping": { "type": "string", "index": "not_analyzed", "included_in_all": false },
                        "match_mapping_type" : "string"
                     } 
                 }
             ]
  }

Hope this makes sense :slight_smile:

#5