generated by Dall-E
Oftentimes finding what you want entails uncovering the right question to ask. Are you using the right phrases? Are you saying the right keywords to trigger a proper response? What if there's some finite, numerical or spatial restraint you have that will help define what it is ultimately that you want?
While these questions about questions can become quickly and tediously existential, today we can fortunately focus on only one field of inquisition - vector search. And most importantly, we will learn how we can make it easier to find what we want. With a quick and painless step inserted into the process of ingesting our data for vector search, we'll be able to provide better results from users posing their nuanced, qualitative queries.
Background
In a previous Search Labs post by Han Xiang Choong, we learn that creating a hypothetical document that would answer a query with enough truthiness would get us very close to finding the right results. We then embed this hypothetical answer and use it to find similar documents that are real and actually answer the question. The hypothetical document would have more similarity to the answer, thus ensuring more accurate search results.
This method can also be used for extracting multiple fields from an entry - not just one particular field designated for embedding - and creating a hypothetical query comprising numbers, booleans, keywords, and rich text as a search target.
Toys aren't all the same, yet they are
As an example, imagine a particular toy that may be part of a larger collection or universe where its name may occur in many places and its faction may also be quite large. Take my favorite toy growing up: the Battle Armor He-Man action figure from the Masters of the Universe cartoon. Those of us growing up in the 1980s may remember the animated series with good pitted against evil with many memorable characters, both villains and heroes.
I had many of the Masters of the Universe action figures, but my favorite was Battle Armor He-Man, as he had a chest piece that would go from perfectly shiny armor to cracked and worn when pressed down. This simple mechanical upgrade made this toy the most desirable in my toy chest. This will have more import as you read on, I promise.
Finding a replacement action figure in my 40's to relive that nostalgia wasn't easy. As one can imagine, there over 200 individual toys, with two separate reboots of the franchise. As the main protagonist, He-Man as the most produced, with numerous iterations and versions to suit the market at the various release times.
So I have a need to find an action figure from a certain release date range, a specific modifier to account for the Battle Armor, and of course, a price range I'm willing to part with. Like most things I share on the internet, one could represent this as a json object. Here's an example of one that I would be willing to purchase, despite it's lack of original packaging:
he_man = {
"product_name": "Battle Armor He-Man",
"product_type": "Action Figure",
"franchise": "Masters of the Universe",
"year_released": 1983,
"condition": "excellent",
"price": 132.99,
"original_packaging": False,
"complete_set": False,
"product_description": "This Battle Armor He-Man figure is an icon
from the original Masters of the Universe line, showcasing the hero in his
legendary battle gear. The rotating chest plate reveals different levels of
damage, making every skirmish more intense. It’s a nostalgic piece that
captures He-Man’s indomitable spirit and looks amazing on display.",
"included_accessories": [
"Battle Axe",
"Battle Sword",
"Weapon Sheath",
"Battle-Damage Chest Piece"
],
}
The Mattel Quandry
While I could search across modern toy sites with just the product name (keyword search), I would inevitably receive results from three different eras, flooding my results with too many technically correct but not accurate results.
generated by Dall-E
I could try searching the product_descriptions with a few phrases to describe my specific choice (vector search), but I'd be ultimately at the mercy of what was included with in the description text. What if it was just filled with aggressive sales copy and had none of the specific attributes of what I was looking for? I could also add some filtering to my search (hybrid search) but again, that's hoping and fine tuning; at this point I am frustrated and ready to give up.
I have the power!
One way to capture the hearts, minds, and ultimately commerce from users searching for products is to anticipate the absolute best possible question that could ever be asked about a specific product. This question would address every single relevant attribute, dictate a price they're willing to pay, and include a few extra nice-to-haves to really make their holiday gift-giving perfect.
We can't expect our users to create these perfect questions and manipulate our filtering capabilities to find the perfect product, but we can create this perfect question ourselves and use it as a vector_embedding to use against their query.
By having an LLM create a query using all of the relevant attributes of a product, then embedding this hypothetical question as a vector, we can capture similar questions that may or may not have all of the attributes that a product has. This allows us to skip filtering and relying on an embedded representation of text that may not adequately include specific product attributes.
This is a relatively straightforward process that inserts an LLM generation of text into an entry that is embedded and used as the main search vector. The results are clear, accurate, and consistent.
Read the code - Creating the perfect question
Lets use Open AI to generate our HyDE documents. Here's a simple Python snippet to contact Open AI and request a completion:
from openai import OpenAI
ai_client = OpenAI()
def get_ai_completion(prompt):
completion = ai_client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": prompt},
],
)
return completion
This is simple enough - the real power comes from the prompt. When we create an entry for a new toy, we create a prompt that has all of the necessary attributes included in the text. This creates a hypothetical question that hits all of our required search parameters for this specific toy.
Here is a simple prompt to generate a query:
prompt = f'Create a detailed query profile written from a customers
perspective describing what they are looking for when shopping for this
exact item. Include key characteristics like type, features, use cases,
quality aspects, materials, specific price range, an appropriate age for
the user, and target user. Focus on aspects a shopper would naturally
mention in their search query.
Format: descriptive text without bullet points or sections.
Example: "Looking for a high-end lightweight carbon fiber road bike for
competitive racing with electronic gear shifting and aerodynamic frame
design suitable for experienced cyclists who value performance and
speed."
Describe this product in natural language that matches how real
customers would search for it. Make sure to mention the target age and a
close approximation of the price to a whole number in the output text
somewhere.
Here are the product details:
Product Name: "{product_name}"
About Product: "{product_description}"
Price: "{price}"
Target Age: "{target_age}"'
And here is a helper function to easily create these prompts:
def generate_prompt(toy):
prompt = f'Create a detailed query profile written from a customers perspective describing what they are looking for when shopping for this exact item. Include key characteristics like type, features, use cases, quality aspects, materials, specific price range, an appropriate age for the user, and target user. Focus on aspects a shopper would naturally mention in their search query. \nFormat: descriptive text without bullet points or sections. \nExample: "Looking for a high-end lightweight carbon fiber road bike for competitive racing with electronic gear shifting and aerodynamic frame design suitable for experienced cyclists who value performance and speed." Describe this product in natural language that matches how real customers would search for it. Make sure to mention the target age and a close approximation of the price to a whole number in the output text somewhere. Here are the product details: \nProduct Name: {toy["product_name"]} \nAbout Product: {toy["product_description"]} \nPrice: {toy["price"]} \nTarget Age: {toy["target_age"]}'
return prompt
Let's read a query generated by our LLM from the above prompt and he-man object:
"I'm on the hunt for a Battle Armor He-Man action figure that really
captures the essence of the original Masters of the Universe line. I'm
ideally looking for a version that showcases He-Man in his legendary battle
gear because I want something that stands out on display and evokes
nostalgia. One of the key features I'm interested in is the rotating chest
plate that reveals different levels of damage; that not only adds a fun
interactive element but also enhances the play experience, making every
skirmish more thrilling for kids who love creative storytelling. \n\nSince
this will be a gift aimed at children aged 7 and up, I'm also considering the
quality aspects of the figure. It should be well-made, durable enough to
withstand rough play, and feature vibrant colors and intricate details that
truly reflect the iconic character's spirit. I want to ensure it's something
that will not just sit on a shelf but will also be played with, sparking
imagination and adventure.\n\nIn terms of price, I’m looking for something
within the $130 range, around $133, as I want to ensure it’s a quality
collector’s item while still being accessible for a young fan. Overall, I want
to find that perfect Battle Armor He-Man figure that balances nostalgia for
older collectors with robustness for kids to enjoy."
The LLM couldn't have described the pure joy I had play with my childhood toy any better!
When indexing a new toy, the step of creating this query is done before the data enters Elasticsearch.
Read the Code - Creating the Index
Index creation using HyDE is no different than any other standard index using vector search. We'll include the raw query profile text as well as the query_profile (embedded vector) for inspection and fine tuning to make sure our questions are how we want them in development. We'll be using Elastic's ELSER v2 model for embedding.
def create_index():
mappings = {
"mappings": {
"properties": {
"product_name": {"type": "text"},
"product_description": {"type": "text"},
"price": {"type": "float"},
"target_age": {"type": "integer"},
"query_profile": {"type": "sparse_vector"},
"raw_query_profile": {"type": "text"},
}
}
}
# Check if the index already exists
if not client.indices.exists(index=index_name):
client.indices.create(index=index_name, body=mappings)
print(f"Index '{index_name}' created successfully.")
else:
print(f"Index '{index_name}' already exists.")
create_index()
Read the Code - Creating an Ingest Pipeline
This pipeline will embed the raw_query_profile text and store it at query_profile as soon as it is added to the index. Note that this can be done with other embedding models as well.
def create_ingest_pipeline():
resp = client.ingest.put_pipeline(
id="elser-v2",
processors=[
{
"inference": {
"model_id": "elser_model_2",
"input_output": [
{
"input_field": "raw_query_profile",
"output_field": "query_profile",
}
],
}
}
],
)
print(resp)
create_ingest_pipeline()
Read the Code - Deploy the embedding model
We'll need to create an inference endpoint to point to when we embed the query_profile as well as the incoming user queries. This creates a stable, named target that we can fine tune later on based on performance and speed.
def create_inference_endpoint():
resp = client.inference.put(
task_type="sparse_embedding",
inference_id="elser_model_2",
inference_config={
"service": "elser",
"service_settings": {"num_allocations": 1, "num_threads": 1},
},
)
print(resp)
create_inference_endpoint()
Read the Code - Add an entry
Now we add a single entry. Note that we'll generate a prompt that contains the specific information of this toy. We'll then send that prompt to our LLM to generate our hypothetical query document. Then we'll have it embedded by Elastic when it is stored.
def add_one_toy(toy):
prompt = generate_prompt(toy)
ai_response = get_ai_completion(prompt)
toy["raw_query_profile"] = ai_response.choices[0].message.content
try:
resp = client.index(index=index_name, body=toy, pipeline="elser-v2")
print(resp)
except IndexError as e:
print(e)
add_toy(he_man)
Read the Code - Query the index
Now lets query our index using the query_profile vectors as the search basis.
toy_query = "I'm looking for the classic He-man action figure toy from the
80s with a muscular build and a sword. He had a neat trick where his chest
would get battle damage if it was pressed down upon. I'm willing to pay
around $100 to $150 dollars. It's uhh... for my 7 year old's birthday.
Sure. That's it."
resp = client.search(
index=index_name,
query={
"sparse_vector": {
"field": "query_profile",
"inference_id": "elser_model_2",
"query": toy_query,
}
},
)
print(resp)
""" TOP THREE RESULTS:
{
"_score": 35.444317,
"_source": {
"product_name": "Battle Armor He-Man",
"target_age": 7,
"price": 132.99
}
},
{
"_score": 33.13795,
"_source": {
"target_age": "7",
"price": "11.99",
"product_name": "Captain Thunder Action Figure"
}
},
{
"_score": 22.948023,
"_source": {
"target_age": "2",
"price": "14.99",
"product_name": "My First Doll"
}
}
"""
This is a very simple, very sparse example of how adding HyDE to your query flow may greatly improve your search results. As many of us have been hunting down the perfect gift this holiday season, getting a little assist from Mr. HyDE - or Vector Search's little helper - may make the entire process faster and a little less of a shot in the dark.
generated by Dall-E
Make sure to look out for the full Search Labs article by Han Xiang Choong on boosting e-commerce results with HyDE enhancements with your vector embedding.
Thank you all for reading through this year's advent of code, and I wish every one of you a wonderful holiday season with minimum failures and maximum human downtime!
- Justin
Repository: GitHub - justincastilla/santas-little-helper: A demonstration of HyDE enhanced vector search