Can i use ELSER and OpenAI Embeddings together in Elasticsearch?

[shakunvohra]

I was wondering if I can use ELSER Vectors and OpenAI Embeddings together in same mapping for a better RAG results. The thought behind this is -
take top 3 results from ELSER and take top 3 from OpenAI embeddings and combine them to send it to a llm for response.

{
"testing": {
"mappings": {
"properties": {
"embedding": {
"type": "dense_vector",
"index": true,
"similarity": "cosine"
},
"metadata": {
"properties": {
"chunk_id": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"id": {
"type": "long"
},
"title": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
"text": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"vector": {
"properties": {
"is_truncated": {
"type": "boolean"
},
"model_id": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"tokens": {
"type": "rank_features"
}
}
}
}
}
}
}

When i index, i get this - elasticsearch.BadRequestError: BadRequestError(400, 'document_parsing_exception', '[1:5223] failed to parse: [rank_features] fields do not support indexing multiple values for the same rank feature [vector.tokens.pr] in the same document')

[Kathleen_DeRusso]

Hey, have you tried using a nested field for your rank_features field? It's possible that you're trying to chunk your documents and index the same feature dimension twice. Check out this search labs blog for more examples on this.

Also, helpful note - you can use Markdown syntax to format your code so it's more readable if you need help!

Good luck!