Elasticsearch autocomplete suggestion on array object

my mapping as follow :

`"skills": {
                    "properties": {
                        "id": {
                            "type": "long"
                        },
                        "skillName": {
                            "type": "text",
                            "fields": {
                                "completion": {
                                    "type": "completion",
                                    "analyzer": "standard",
                                    "preserve_separators": false,
                                    "preserve_position_increments": true,
                                    "max_input_length": 50
                                },
                                "keyword": {
                                    "type": "keyword"
                                }
                            }
                        }
                    }
                }`

sample doc :

`"skills": [
                        {
                            "id": 28,
                            "skillName": "auto matics"
                        },
                        {
                            "id": 29,
                            "skillName": "auto parts"
                        }
                    ]`

question : if i am type auto then i want auto parts and auto matics as suggestions.

my suggestion query as follow

{
    "_source": "suggest",
    "suggest": {
        "skillName_suggest": {
            "text": "auto",
            "completion": {
                "field": "skills.skillName.completion",                
                "size": 10,
                "fuzzy": {
                    "fuzziness": 2
                }
            }
        }
    }
}

but i am getting only "auto matics" in response. i want both "auto parts" and "auto matics".

{
    "took": 1,
    "timed_out": false,
    "_shards": {
        "total": 2,
        "successful": 2,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 0,
            "relation": "eq"
        },
        "max_score": null,
        "hits": []
    },
    "suggest": {
        "skillName_suggest": [
            {
                "text": "auto",
                "offset": 0,
                "length": 4,
                "options": [
                    {
                        "text": "auto matics",
                        "_index": "scholarsindex",
                        "_id": "AWuRd4UB7nUhhZ090QzX",
                        "_score": 2.0,
                        "_source": {}
                    }
                   
                ]
            }
        ]
    }
}

Hi @ysunil702

The "options" array in the return will represent the document that matches the search term. In your example you have only one document that can be suggested if the entry is "auto matics" or "auto parts".

If you want to have 2 items in the "options" array you would have to have another document with the "skillName" starting with the term "auto".

1 Like

Thank you for prompt reply @RabBit_BR.
Got it. one more scenario I had passed text 'matics' or 'parts' then didn't get suggestion. if i have pass matics then i want 'auto matics' in response.
Is it possible or have any other mapping settings or any other method to achieve this?

For this scenario I used the search-as-you-type. But I had change the mapping, skills makes a nested type.

PUT my-index-000001
{
  "mappings": {
    "properties": {
      "skills": {
        "type": "nested", 
        "properties": {
          "id": {
            "type": "long"
          },
          "skillName": {
            "type": "text",
            "fields": {
              "completion": {
                "type": "search_as_you_type"
              },
              "keyword": {
                "type": "keyword"
              }
            }
          }
        }
      }
    }
  }
}

Doc

POST my-index-000001/_doc
{
  "skills": [
    {
      "id": 28,
      "skillName": "auto matics"
    },
    {
      "id": 29,
      "skillName": "auto parts"
    }
  ]
}

Query

GET my-index-000001/_search?filter_path=hits.hits.inner_hits.skills.hits.hits
{
  "_source": false, 
  "query": {
    "nested": {
      "path": "skills",
      "query": {
        "multi_match": {
          "query": "matics",
          "type": "bool_prefix",
          "operator": "and", 
          "fields": [
            "skills.skillName.completion",
            "skills.skillName.completion._2gram",
            "skills.skillName.completion._3gram"
          ]
        }
      }, "inner_hits": {}
    }
  }
}

Wonderful @RabBit_BR thank you so much for your efforts.
But after changing in mapping my earlier search query not working for skills.

following as my search query

{"sort":[{"createdOn":"desc"}],"from":0,"size":5,"query":{"bool":{"must":[{"multi_match":{"query":"auto cad","fields":["skills.skillName.completion^9.0","specialization^8.0","bio^7.0","collegeOrUniversity^6.0","location^5.0"]}}],"must_not":[],"should":[]}}}

if you choose the nested strategy you must use Nested Query.

The query would be:

{
  "from": 0,
  "size": 5,
  "query": {
    "bool": {
      "should": [
        {
          "nested": {
            "path": "skills",
            "query": {
              "match": {
                "skills.skillName": {
                  "query": "auto",
                  "boost": 9
                }
              }
            }
          }
        },
        {
          "multi_match": {
            "query": "auto",
            "fields": [
              "specialization^8.0",
              "bio^7.0",
              "collegeOrUniversity^6.0",
              "location^5.0"
            ]
          }
        }
      ]
    }
  }
}
1 Like

Yeah its working fantastic. thank you @RabBit_BR
In my earlier query if i am pass text 'autocad' it will give response with 'auto cad' and 'autocad' but now i am getting only autocad.
can you give me suggestion of this?

provide the documents with "autocad" and "auto cad"

I want both (autocad and auto cad) in search query response if i am pass autocad or auto cad
following are the two docs:
(1)

"skills": [  
                        {
                            "id": 26,
                            "skillName": "auto cad"
                        },
                        {
                            "id": 27,
                            "skillName": "database management"
                        }
                    ]

(2)

"skills": [
                        {
                            "id": 10,
                            "skillName": "mechanical engineering"
                        },
                           {
                            "id": 18,
                            "skillName": "autocad"
                        }
                    ]

Following is the search query


{"sort":[{"createdOn":"desc"}],"from":0,"size":5,"query":{"bool":{"must":[],"must_not":[],"should":[{"nested":{"path":"skills","query":{"match":{"skills.skillName":{"query":"autocad","boost":9}}}}},{"multi_match":{"query":"autocad","fields":["specialization^8.0","bio^7.0","collegeOrUniversity^6.0","location^5.0"]}}]}}}

Try replace skills.skillName tposkills.skillName.completion.

Hi @RabBit_BR
I had tried following query but not work.

{
    "from": 0,
    "size": 5,
    "query": {
        "bool": {
            "should": [
                {
                    "nested": {
                        "path": "skills",
                        "query": {
                            "match": {
                                "skills.skillName.completion": {
                                    "query": "autocad",
                                     "boost" : 9
                                }
                            }
                        }
                    }
                },
                {
                    "multi_match": {
                        "query": "auto cad",
                        "fields": [
                            "specialization^8.0",
                            "bio^7.0",
                            "collegeOrUniversity^6.0",
                            "location^5.0"
                        ]
                    }
                }
            ]
        }
    }
}

with following mapping

"skills": {
                    "type": "nested",
                    "properties": {
                        "id": {
                            "type": "long"
                        },
                        "skillName": {
                            "type": "text",
                            "fields": {
                                "completion": {
                                    "type": "search_as_you_type",
                                    "doc_values": false,
                                    "max_shingle_size": 3,
                                    "analyzer": "standard"
                                },
                                "keyword": {
                                    "type": "keyword"
                                }
                            }
                        }
                    }
                },

I reproduced your old mapping and query and couldn't get "autocad" and "auto cad" with the search term "autocad".
I believe that the other fields of the document have one of the terms and that's why you have the results but searching for the field "skills.skillName" I didn't get a result with the old mapping as with the new one.

1 Like

Here you can see full information regarding auto cad query, mapping and response.

Old mapping


 "skills": {
                    "properties": {
                        "id": {
                            "type": "long"
                        },
                        "skillName": {
                            "type": "text",
                            "fields": {
                                "completion": {
                                    "type": "completion",
                                    "analyzer": "standard",
                                    "preserve_separators": false,
                                    "preserve_position_increments": true,
                                    "max_input_length": 50
                                },
                                "keyword": {
                                    "type": "keyword"
                                }
                            }
                        }
                    }
                },

Query:
{"sort":[{"updatedOn":"asc"}],"from":0,"size":5,"query":{"bool":{"must":[{"multi_match":{"query":"autocad","fields":["title^9.0","skills.skillName.completion^8.0","description^7.0","university^6.0","fieldOfStudy^5.0"]}}],"must_not":[x]}}}

And full response:


{
    "took": 1,
    "timed_out": false,
    "_shards": {
        "total": 2,
        "successful": 2,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 2,
            "relation": "eq"
        },
        "max_score": null,
        "hits": [
            {
                "_index": "scholarsindex",
                "_id": "6GvWgYUB7nUhhZ09jww1",
                "_score": null,
                "_source": {
                    "id": 3,
                    "firstName": "Aliakbar",
                    "lastName": "Baldiwala",
                    "courseName": null,
                    "location": "Canada",
                    "collegeOrUniversity": "University of Ottawa",
                    "bio": "As the founder and CEO of The Scholars Club Institution Inc and OATS Institute, I bring a unique blend of entrepreneurship and technical expertise to the company. With a background in Mechanical engineering, Teaching experience, and over 5 years of experience in the tech industry, I have a deep understanding of the challenges and opportunities that start-ups face today.\n\nSince launching The Scholars Club Institution Inc, in 2022 (Canada & USA), I have grown the company from a small team of passionate individuals to a thriving business with a global customer base from more than 38 countries.\nI am passionate about using technology to solve real-world problems and making a positive impact in the world. I am a firm believer in the power of collaboration and teamwork, and I am committed to building a culture of innovation and excellence at The Scholars Club Institution Inc.\nIn my spare time, I enjoy creating new innovative designs, watching anime, Reading Sci-Fi and space exploration books, and theorizing about the possibilities of space exploration. I hold a M.A.Sc degree in Mechanical Engineering from the University of Ottawa, Canada.",
                    "fieldOfStudy": "Mechanical engineering",
                    "levelOfStudy": 3,
                    "profileImage": "https://thescholarsclub.blob.core.windows.net/scholarsclub/user/3/Profile%20photo.png",
                    "createdOn": "2023-01-02T01:11:19.2656358",
                    "updatedOn": "2023-01-02T01:25:14.5449286",
                    "proposed": 1,
                    "jobCompleted": 0,
                    "experienceLevel": 3,
                    "rating": 0.0,
                    "specialization": "Mechanical Engineering",
                    "skills": [
                        {
                            "id": 10,
                            "skillName": "mechanical engineering"
                        },
                        {
                            "id": 8,
                            "skillName": "ic engine"
                        },
                        {
                            "id": 16,
                            "skillName": "thermodynamics"
                        },
                        {
                            "id": 17,
                            "skillName": "solidworks"
                        },
                        {
                            "id": 18,
                            "skillName": "autocad"
                        },
                        {
                            "id": 19,
                            "skillName": "dynamics of machine"
                        },
                        {
                            "id": 7,
                            "skillName": "automobile engineering"
                        },
                        {
                            "id": 20,
                            "skillName": "microsoft ppt"
                        },
                        {
                            "id": 13,
                            "skillName": "designing"
                        },
                        {
                            "id": 21,
                            "skillName": "engine design"
                        },
                        {
                            "id": 22,
                            "skillName": "drafting"
                        },
                        {
                            "id": 23,
                            "skillName": "fluid dynamics"
                        },
                        {
                            "id": 24,
                            "skillName": "mechanics of solids"
                        },
                        {
                            "id": 25,
                            "skillName": "thesis writing"
                        }
                    ]
                },
                "sort": [
                    1672622714544
                ]
            },
            {
                "_index": "scholarsindex",
                "_id": "6WvWgYUB7nUhhZ09jww1",
                "_score": null,
                "_source": {
                    "id": 1,
                    "firstName": "Hemang",
                    "lastName": "Rathod",
                    "courseName": null,
                    "location": "India",
                    "collegeOrUniversity": "Sardar Patel University",
                    "bio": "A Full-Stack Developer writes functional, clean code on the front - and back-end. You must know how to collaborate with product managers and development teams to ideate software solutions. You will be in charge of software debug, troubleshoot, and upgrade. Full-Stack Developer’s other responsibilities include testing software to ensure efficiency and responsiveness, writing efficient APIs, making security and data protection settings, and writing technical documentation. You work with analysts and data scientists to enhance software performance. You need to create applications and features with a mobile responsive design.The skills and requirements to be a Full-Stack Developer include knowledge in different back-end languages like Java, Python as well as JavaScript frameworks such as React, Angular, and Node.js. It is also important if you’re familiar with web servers, databases, and UI/UX design. You must also have experience in developing mobile and desktop applications. As a Full-Stack Developer, you must have great attention to detail, good teamwork and communications skills, and organizational skills. You must have proven experience as a Full Stack Developer or the same role. ",
                    "fieldOfStudy": "Computer Engineering Technologies/Technicians",
                    "levelOfStudy": 3,
                    "profileImage": "https://thescholarsclub.blob.core.windows.net/scholarsclub/user/1/batman.png",
                    "createdOn": "2022-12-30T13:29:19.9170016",
                    "updatedOn": "2023-01-04T05:05:04.1090407",
                    "proposed": 3,
                    "jobCompleted": 0,
                    "experienceLevel": 2,
                    "rating": 0.0,
                    "specialization": "Fullstack Developer",
                    "skills": [
                        {
                            "id": 1,
                            "skillName": "react"
                        },
                        {
                            "id": 2,
                            "skillName": "next"
                        },
                        {
                            "id": 3,
                            "skillName": "c#"
                        },
                        {
                            "id": 4,
                            "skillName": ".net"
                        },
                        {
                            "id": 5,
                            "skillName": "angular"
                        },
                        {
                            "id": 6,
                            "skillName": "power plateform"
                        },
                        {
                            "id": 26,
                            "skillName": "auto cad"
                        },
                        {
                            "id": 27,
                            "skillName": "database management"
                        }
                    ]
                },
                "sort": [
                    1672808704109
                ]
            }
        ]
    }
}

You used "preserve_separators": false. There isnt this parameter in "search_as_you_type".

I created a new analyzer and field for this scenario. This analyzer "no_separators_analyzer" will remove whitespace and it is used in the field noSeparators.

PUT idx_test
{
  "settings": {
    "analysis": {
      "analyzer": {
        "no_separators_analyzer": {
          "filter": [
            "lowercase",
            "whitespace_remove"
          ],
          "tokenizer": "keyword"
        }
      },
      "filter": {
        "whitespace_remove": {
          "type": "pattern_replace",
          "pattern": " ",
          "replacement": ""
        }
      }
    }
  },
  "mappings": {
    "properties": {
      "skills": {
        "type": "nested",
        "properties": {
          "id": {
            "type": "long"
          },
          "skillName": {
            "type": "text",
            "fields": {
              "completion": {
                "type": "search_as_you_type",
                "doc_values": false,
                "max_shingle_size": 3,
                "analyzer": "standard"
              },
              "noSeparators": {
                "type": "text",
                "analyzer": "no_separators_analyzer"
              },
              "keyword": {
                "type": "keyword"
              }
            }
          }
        }
      }
    }
  }
}

POST idx_test/_doc
{
  "skills": [
    {
      "id": 26,
      "skillName": "auto cad"
    },
    {
      "id": 27,
      "skillName": "database management"
    }
  ]
}

POST idx_test/_doc
{
  "skills": [
    {
      "id": 10,
      "skillName": "mechanical engineering"
    },
    {
      "id": 18,
      "skillName": "autocad"
    }
  ]
}

Query

GET idx_test/_search
{
  "from": 0,
  "size": 5,
  "query": {
    "bool": {
      "should": [
        {
          "nested": {
            "path": "skills",
            "query": {
              "multi_match": {
                "query": "autocad",
                "fields": [
                  "skills.skillName.completion",
                  "skills.skillName.noSeparators"]
              }
            }
          }
        },
        {
          "multi_match": {
            "query": "autocad",
            "fields": [
              "specialization^8.0",
              "bio^7.0",
              "collegeOrUniversity^6.0",
              "location^5.0"
            ]
          }
        }
      ]
    }
  }
}
1 Like

Thank you so much @RabBit_BR for your valuable response. its working good as expected.

1 Like