Query_string on nested objects

Elastic Stack Elasticsearch
1

Hello there !

I have a problem with nested objetcts :smiley:

Field :

 
"listSecteurs":
{
    "type" : "nested",
    "include_in_parent": true,
    "properties" : 
    {
        "secteurName" : {"type": "string", "analyzer": "analyzerFr"},
        "secteurExp" : {"type": "integer"}
    }
},

Values :

 "listSecteurs": [
    {
      "secteurName": "Ferroviaire",
      "secteurExp": "4"
    },
    {
      "secteurName": "qwr",
      "secteurExp": "3"
    }
],

Search query (php api) :

 
 $params = [
     'index' => 'index-fr',
     'type' => 'candidat',
     'body' => [
         'size' => 20,
         'query' => [
             'query_string' => [
                 'analyze_wildcard' => true,
                 'default_operator' => 'AND',
                 'query' => [
                     'fields' => [ 'listSecteurs'  ],
                     'query' => qwr
                 ]
             ]
         ]
     ]
 ];

This search query doesn't find the "qwr" in my value :smile:

2

If you are searching for attributes on nested documents you will need to use the nested query.

This chapter of the 'Elasticsearch:The Definitive Guide` book would be worth a read to understand how nested objects work in Elasticsearch.

3

Thanks !

"nested": {
    "path" : ["listSecteurs"],
    "query": {
      "query_string": {
        "fields": ["listSecteurs.secteurName"],
        "query": "qwr"
      }
    }
  },

But i want to search on other fields and they're simple string fields. I need to use query_string (for the AND/OR), but i can't manage to combine the nested and the query string for simple fields.

1 Like
4

You can combine the nested properties into a leaf-level equivalent of the "_all" field that sits at the root. See the "skills.summary" in the example below:

PUT test
{
   "settings": {
	  "number_of_replicas": "0",
	  "number_of_shards": "1"
   },
   "mappings": {
	  "candidate": {
		 "dynamic": "strict",
		 "properties": {
			"name": {
			   "type": "string"
			},
			"skills": {
			   "type": "nested",
			   "properties": {
				   "summary":{
					   "type":"string"
				   },
				  "sector": {
					 "type": "string",
					  "copy_to": "skills.summary"
				  },
				  "experience": {
					 "type": "string",
					  "copy_to": "skills.summary"
				  }
			   }
			}
		 }
	  }
   }
}

POST test/candidate
{
	"name":"mark",
	"skills":[
		{
			"sector":"lucene",
			"experience":5
		},
		{
			"sector":"javascript",
			"experience":2
		}
	
	]
}

GET test/candidate/_search
{
   "query": {
	  "nested": {
		 "path": "skills",
		 "query": {
			"query_string": {
				"default_field": "skills.summary",
			   "query": "lucene AND 5"
			}
		 }
	  }
   }
}
5

Thanks, i actually do almost the same thing :

 "mappings" :
{
    "candidat": 
    {
        "properties": 
        {
            "usersFunction" : {"type": "string"},
            "listOutils" : 
            {
                "type" : "nested",
                "properties" : 
                {
                    "outilName" : {"type": "string"},
                    "outilRating" : {"type": "integer"}
                }
            },
        }  
    }  
}

With the nested search i can search inside listOutils, but i dont know how to search in both listOutils et usersFunction (skills.summary and name for your example)

This doesn't work

"query": {
    "nested": {
        "path" : ["listOutils"],
        "query": {
          "query_string": {
            "fields": ["listOutils.outilName"],
            "query": "word excel"
          }
        }
    },
    "query_string": {
        "analyze_wildcard": true,
        "default_operator": "AND",
        "query":  {
          "fields": ["usersFunction"],
          "query": "word excel"
        }
    }
  }
6

Inviting users to use AND/OR () type logic using query_string may seem appealing but the query parser lacks the syntax to express nested clauses and even if it did I'm not sure most end users would know how to use it or want to.
It's a potentially dangerous combination to offer users the tools to write formal Boolean logic but with no clear way of expressing the logic about which values they are ANDing etc.

7

Don't worry, our clients know how to use the boolean and want them :smiley:

We had the AND/OR, parenthesis and double quote in a home made search engine but Es if way more efficient.

So i can't search inside both simple string field and nested object ?

8

The problem is the user needs to be explicit about which context they are AND or ORing in. Are they ANDing terms that come from the same nested object or are they properties they hope to find in different objects.

query_string does not let them express that context.

9

Sorry i dont understand what you said.

When a user search, he will write "word AND excel". I will take this search query and match fields who contains word and excel.

It works with :

'size' => 20,
'query' => [
    'query_string' => [
        'query' => [
            'fields' => [ 'usersFunction', other string field],
            'query' => "word AND excel"
        ]
    ]
]

listOutils.outilName won't work there (because it's a nested object), so i have to use the nested to search.

But if i use nested like this :

"size": 20, 
"nested": {
    "path" : ["listOutils"],
    "query": {
      "query_string": {
        "fields": ["listOutils.outilName"],
        "query": "Chef de projet éléctronique"
      }
    }
  }

I can't perform a search on other string fields :confused:
That's why i ask if i can do both in one query, to search in the string fieds and nested objects.

10

If your query is word AND excel I assume the user is finding a candidate with both of these skills. That would be easily achieved having a "flat" mapping and not using nested at all.

The reason I expect you opted for nested mappings in your original example is that at some point someone wanted to query for secteurName: qwr AND secteurExp:3. In this case nested mappings are necessary to avoid the "cross-matching" problem of flattening all skills into a single Lucene document.

However - the 2 queries word AND excel and qwr AND 3 have very different assumptions about how matching should work. The first assumes you are querying across nested docs and the second assumes you are querying inside nested docs and there is no way for the user to control which behaviour they require using the query_string query parser.

If your users are only doing cross-object ANDing like the word AND excel example then consider using copy_to to put content in the root or drop the use of nested if you have no other need for it.

11

Right, i understand now.

And yes, your're right, the user wants to search word and excel but also search some specific skill by name and experience (word 3years).

In the end, i can do like your exemple or i can have nested object and flat array in my mapping and choose which one to use.

Thanks ! :smile:

12

Maybe run both? You can of course have a bool should array with the user query in a nested context and also as a flattened non-nested one. The doc that matches the nested clause will also match the flattened clause and come top. The doc that matches only the flattened clause should not rank as highly.