Keyword Query Does not Work

[Ibrahim_ozgon (Ibrahim özgön)]

Hello everyone,

I have a problem with keyword query.

I want to filter categoryProperties by key and value.

key is"color" and value contains "mavi"

But it gives me document it contains key is "color" and value contains "Beyaz"

Do you have any idea why?

REQUEST

{
	"from": 0,
	"size": 50,
	"sort": [{
		"id": {
			"order": "asc"
		}
	}],
	"query": {
		"bool": {
			"must": [{
				"nested": {
					"query": {
						"bool": {
							"must": [{
								"nested": {
									"query": {
										"bool": {
											"should": [{
												"bool": {
													"must_not": [{
														"term": {
															"searchQuery.categoryProperties.key": {
																"value": "color"
															}
														}
													}]
												}
											},
											{
												"bool": {
													"must": [{
														"term": {
															"searchQuery.categoryProperties.key": {
																"value": "color"
															}
														}
													},
													{
														"nested": {
															"query": {
																"term": {
																	"searchQuery.categoryProperties.values.value": {
																		"value": "Mavi"
																	}
																}
															},
															"path": "searchQuery.categoryProperties.values"
														}
													}]
												}
											}]
										}
									},
									"path": "searchQuery.categoryProperties"
								}
							}]
						}
					},
					"path": "searchQuery"
				}
			}]
		}
	}
}

RESPONSE

{
	"took": 32,
	"timed_out": false,
	"_shards": {
		"total": 5,
		"successful": 5,
		"failed": 0
	},
	"hits": {
		"total": 1,
		"max_score": null,
		"hits": [{
			"_index": "favoritesearchsearchmodelindex_2",
			"_type": "favoritesearchsearchmodel",
			"_id": "76175",
			"_score": null,
			"_source": {
				"searchQuery": {
					"categoryProperties": [{
						"key": "maxkm",
						"values": [{
							"value": "1200000"
						}],
						"intValue": 1200000
					},
					{
						"key": "color",
						"values": [{
							"value": "Beyaz"
						}]
					},
					{
						"key": "gear",
						"values": [{
							"value": "Otomatik"
						}]
					},
					{
						"key": "category",
						"values": [{
							"value": "26553"
						}],
						"intValue": 26553
					}],
					"hasPhoto": false,
					"currency": 0,
					"city": [],
					"category": [{
						"id": 26553
					}]
				}
			}
		}]
	}
}

MAPPING

{
	"favoritesearchsearchmodelindex_2": {
		"mappings": {
			"favoritesearchsearchmodel": {
				"properties": {
					"searchQuery": {
						"type": "nested",
						"properties": {
							"categoryProperties": {
								"type": "nested",
								"properties": {
									"intValue": {
										"type": "integer"
									},
									"key": {
										"type": "keyword"
									},
									"values": {
										"type": "nested",
										"properties": {
											"value": {
												"type": "keyword"
											}
										}
									}
								}
							}
						}
					}
				}
			}
		}
	}
}

[Mark_Harwood]

Not a comment on why this is going wrong but more of a general observation - you may have gone overboard with the use of "nested". If you have an array of objects and they only have one property (like your "values" example) then there is no benefit to mapping as nested instead of the simplerobject type. The use of nested is required to preserve associations between multiple fields in the same object by using additional Lucene documents to hold the related values together.
This is an old slide deck [1] that outlines the use case for nested - the implementation details differ a little in elasticsearch but the rationale for when you need nested docs is the same and it should give an indication of the overhead they add to disk storage and query complexity.

[1] https://www.slideshare.net/MarkHarwood/proposal-for-nested-document-support-in-lucene

[Ibrahim_ozgon (Ibrahim özgön)]

Thank you for your response Mark. I didn't know that. I will change them to object asap.

[Ibrahim_ozgon (Ibrahim özgön)]

Thank you again @Mark_Harwood.
According to your response, I changed my mapping.
New Mapping

{
	"favoritesearchsearchmodelindex_2": {
		"mappings": {
			"favoritesearchsearchmodel": {
				"properties": {
					"searchQuery": {
						"type": "nested",
						"properties": {
							"categoryProperties": {
								"properties": {
									"key": {
										"type": "keyword"
									},
									"numberValue": {
										"type": "double"
									},
									"values": {
										"properties": {
											"value": {
												"type": "keyword"
											}
										}
									}
								}
							},
							"keyList": {
								"properties": {
									"value": {
										"type": "keyword"
									}
								}
							}
						}
					}
				}
			}
		}
	}
}

After changing mapping i realized that;
I'm searching for searchQuery.categoryProperties.key is not color. I have an array and if one of the key is not colorits ok for search, but not for me.
I created a keyList array and put all grouped keys of searchQuery.categoryProperties.key to keyListobject.
Now I'm searching for keyListfirst. It gives me the correct response. This resolved my problem.

Here is the correct REQUEST

{
	"query": {
		"bool": {
			"must": [{
				"nested": {
					"query": {
						"bool": {
							"filter": [{
								"bool": {
									"should": [{
										"bool": {
											"must_not": [{
												"term": {
													"searchQuery.keyList.value": {
														"value": "color"
													}
												}
											}]
										}
									},
									{
										"bool": {
											"must": [{
												"term": {
													"searchQuery.categoryProperties.key": {
														"value": "color"
													}
												}
											},
											{
												"term": {
													"searchQuery.categoryProperties.values.value": {
														"value": "Mavi"
													}
												}
											}]
										}
									}]
								}
							}]
						}
					},
					"path": "searchQuery"
				}
			}]
		}
	}
}

With @Mark_Harwood help, i don't need a lot of nested queries!