Introduction
We've all played with those song generators out there on the internet to generate songs. More recently we've used LLMs such as ChatGPT to generate songs or poems in the style of a particular artist. But what if we could combine these things with lyrics stored in our own Elasticsearch cluster?
In today's entry, we'll walk through how to build a Christmas song generator using Elasticsearch, LangChain and OpenAI LLM.
Architecture
Our generator, for which the code is available in this GitHub repo, is modelled on a similar architecture used for most vanilla websites:
We have 3 layers:
- A simple framework independent HTML, JavaScript and CSS UI (
index.html|js|css). - Node.js Express server (
server/server.js) with utilities for communication between Elasticsearch and OpenAI LLM using the LangChain JavaScript library. - Elasticsearch data store, containing song lyrics extracted from http://www.christmassongs.net/song-lyrics via the Elastic Web Crawler and transformed using an Ingest pipeline.
Lyrics
The key elements we want from our ingested documents are the song title and lyrics to pass to our LLM. With the page structure used on this site, we can make use of CSS selector logic in the web crawler to create new fields song-title (as title is a reserved field for the title of the page) and lyrics:
As you can see from the above rules, the lyrics are pulled from the first <p> child tag of the element with the id lyrics and the title is pulled from the <h1> tag with class lyrics, giving us the following fields:
{
// Other fields omitted
"last_crawled_at": "2023-11-24T13:58:34Z",
"song-title": "White Christmas Lyrics",
"title": "White Christmas Lyrics | ChristmasSongs.net",
"url": "http://www.christmassongs.net/lyrics/white-christmas",
"lyrics": "I'm dreaming of a white Christmas Just like the ones I used to know Where the treetops glisten And children listen To hear sleigh bells in the snow. I'm dreaming of a white Christmas With every Christmas card I write May your days be merry and bright And may all your Christmases be white. I'm dreaming of a white Christmas With every Christmas card I write May your days be merry and bright And may all your Christmases be white."
}
From here we want to transform the documents to remove the trailing Lyrics value, delete any documents from surplus pages in our crawl that don't have the song-title and lyrics fields, as well as generate vectors using our previously uploaded text embedding model. This can be done using the below ingest pipeline:
PUT _ingest/pipeline/search-christmas-songs-vector-embedding-pipeline
{
"processors": [
{
"drop": {
"if": "ctx['song-title'] == null && ctx['lyrics'] == null"
}
},
{
"gsub": {
"field": "song-title",
"pattern": """ Lyrics""",
"replacement": ""
}
},
{
"inference": {
"model_id": "sentence-transformers__msmarco-minilm-l-12-v3",
"input_output": [
{
"input_field": "lyrics",
"output_field": "lyrics_embedding.predicted_value"
},
{
"input_field": "song-title",
"output_field": "song_title_embedding.predicted_value"
}
]
}
}
]
}
With a simple reindex:
POST _reindex
{
"source": {
"index": "search-christmas-songs"
},
"dest": {
"index": "vector-search-christmas-songs",
"pipeline": "search-christmas-songs-vector-embedding-pipeline"
}
}
We end up with documents similar to the ones below that can be used by the song generator:
{
// Other fields omitted
"last_crawled_at": "2023-11-24T13:58:34Z",
"song-title": "White Christmas",
"title": "White Christmas Lyrics | ChristmasSongs.net",
"url": "http://www.christmassongs.net/lyrics/white-christmas",
"lyrics_embedding": {
"predicted_value": [
-0.016469480469822884,
0.02412194199860096,
// Other values omitted
],
"model_id": "sentence-transformers__msmarco-minilm-l-12-v3"
},
"lyrics": "I'm dreaming of a white Christmas Just like the ones I used to know Where the treetops glisten And children listen To hear sleigh bells in the snow. I'm dreaming of a white Christmas With every Christmas card I write May your days be merry and bright And may all your Christmases be white. I'm dreaming of a white Christmas With every Christmas card I write May your days be merry and bright And may all your Christmases be white.",
"song_title_embedding": {
"predicted_value": [
0.0614176020026207,
0.29176318645477295,
// Other values omitted
]
}
}
Note that we would recommend not storing the embeddings in the source document. I've left them in here for debugging and to show the document structure easily.
Search
When a user enters and submits their values into the generator UI, we need to fetch a Christmas song to pass to OpenAI. Using the Elasticsearch JavaScript client we can connect to our Elastic Cloud deployment using a Cloud ID and API Key:
const elasticsearch = require("@elastic/elasticsearch");
const cloudID = process.env.ELASTIC_CLOUD_ID;
const apiKey = process.env.ELASTIC_API_KEY;
const index = "vector-search-christmas-songs";
const client = new elasticsearch.Client({
cloud: { id: cloudID },
auth: { apiKey: apiKey },
});
In this case, multiple knn, or k-nearest neighbour searches against our title and lyrics vectors are blended with a multi_match query using reciprocal rank fusion, or RRF, to try and find a similar song where we don't have the specified title in our index:
async function getTopDocumentsForSongTitle(title) {
if (!client) {
throw new Error("Unable to connect to Elasticsearch")
}
return client.search({
index: index,
fields: ["song-title", "lyrics"],
query: {
multi_match: {
query: title
}
},
knn: [
{
field: "song_title_embedding.predicted_value",
k: 1,
num_candidates: 100,
query_vector_builder: {
text_embedding: {
model_id: "sentence-transformers__msmarco-minilm-l-12-v3",
model_text: `A Christmas song with a title that is similar to ${title}`
}
}
},
{
field: "lyrics_embedding.predicted_value",
k: 1,
num_candidates: 100,
query_vector_builder: {
text_embedding: {
model_id: "sentence-transformers__msmarco-minilm-l-12-v3",
model_text: `A Christmas song with similar lyrics to ${title}`
}
}
}
],
rank: {
rrf: {}
}
});
}
module.exports = { getTopDocumentsForSongTitle }
LangChain LLM Prompt
Once we have obtained the document containing our song from Elasticsearch, it's time to submit a request to the LLM to get the (possibly) best Christmas song ever! LangChain gives our capabilities to connect to the LLM with our own OpenAI API key, with the below snippet:
const openai = require("langchain/chat_models/openai");
const apiKey = process.env.OPENAI_API_KEY;
const chatModel = new openai.ChatOpenAI({
openAIApiKey: apiKey,
temperature: 0.9
});
Notice I have specified a high temperature value of 0.9 to introduce more randomness into the output. This is higher than the LangChain default of 0.7. In some use cases, lower randomness may be more appropriate to make results slightly more predictable, but for a song generator, a bit of randomness doesn't hurt.
Prompts are a complex field, with an entire engineering discipline emerging to handle them. Here I've given the LLM a system message with context on its purpose. The formatMessages call then allows me to add the relevant values from the user to my prompts before receiving the response via predictMessages:
const chatPrompts = prompts.ChatPromptTemplate.fromMessages([
["system", "You are a Christmas song generator that takes user input and generates a song in the style of the provided song"],
["human", "Write a {adjective} Christmas song about {subject}"],
["human", "Include at least one reference to eating {food} and receiving {gift}"],
["human", "Write the song in the style of these lyrics: {lyrics}"],
]);
const formattedChatPrompts = await chatPrompts.formatMessages({
subject: subject,
adjective: adjective,
food: food,
gift: gift,
lyrics: lyrics
});
const messages = await chatModel.predictMessages(formattedChatPrompts);
return messages.content;
Once the content has been returned from the LLM, we can pass it back to the user in a response just like any other content. Check out the GitHub repo for the full implementation.
The Ultimate Christmas Song!
So what is the ultimate Christmas song? Taking our generator for a spin with our favourite mascot Elkie, whose favourite song is White Christmas, we get something a bit like this:
Seasons Greetings!