Vertex AI Gemini Embeddings Task Type Support in Inference API

Elastic Stack Elasticsearch
1

I'm working on implementing semantic search using Vertex AI Gemini embedding models (gemini-embedding-001) through the Inference API. While
the Create a Google Vertex AI inference endpoint | Elasticsearch API documentation shows support for task_type parameter, there appears to be a
difference between what Elasticsearch supports and what Vertex AI offers.

Elasticsearch Vertex AI Inference API supports these task types:

  • rerank
  • text_embedding
  • completion
  • chat_completion

However, Gemini embedding API provides task types optimized for specific use cases:

  • SEMANTIC_SIMILARITY - Embeddings optimized to assess text similarity
  • CLASSIFICATION - Embeddings optimized to classify texts
  • CLUSTERING - Embeddings optimized to cluster texts
  • RETRIEVAL_DOCUMENT - Embeddings optimized for document search
  • RETRIEVAL_QUERY - Embeddings optimized for search queries
  • CODE_RETRIEVAL_QUERY - Embeddings optimized for code block retrieval via natural language
  • QUESTION_ANSWERING - Embeddings for Q&A systems
  • FACT_VERIFICATION - Embeddings for fact checking systems

Questions

  1. Will Elasticsearch's Vertex AI inference endpoint support Google's specific task types (like RETRIEVAL_DOCUMENT, RETRIEVAL_QUERY, etc.) in future releases? These
    task types can impact embedding quality for specific use cases.
  2. How does the current text_embedding task type map to Google's task types? Does it default to a specific Gemini task type, or does it use Google's default
    behavior?

For now, I’m planning to:

  • Use Vertex AI's client libraries directly to generate embeddings with the appropriate task types
  • Index the resulting embeddings via Bulk API
  • Update embeddings manually when needed, outside of the inference pipeline setup

Has anyone else encountered this? Are there alternative approaches or upcoming features that might address this?

Thanks for any insights!

Additional Details:

  • Using Elasticsearch 8.17.3
  • Gemini model: gemini-embedding-001
  • Current embedding dimensions: 3072