I am experiencing an issue with Elasticsearch version 8.18.2 on a self-managed setup. I've set up an index with vector embeddings, templated as dense vectors. The index contains thousands of documents and search operations work correctly. However, when I create a backup using the snapshot functionality and attempt to restore it, the recovered index is empty. The recovery of non-vector-based indexes works as expected.
Steps to Reproduce:
-
Environment Details:
- Elasticsearch Version: 8.18.2
- Setup Type: Self-managed
- Node: AKS (Azure Kubernetes)
-
Index Details:
- Type: Vector embeddings
- Mapping:
"vector": {
"type": "dense_vector",
"dims": 512
}
-
Snapshot Process:
- Repository Type: AWS S3
- Steps to Create Snapshot: Closely followed the official doc
- Verification: Not 100% sure to confirm snapshot contains the index
-
Restore Process:
- Steps to Restore: Closely followed the official doc
- Observations: The index is present but empty post-restore
Expected Behavior:
The restored index should contain all documents and data identical to the state during the snapshot.
Actual Behavior:
The restored index is present in Elasticsearch but contains no documents.
Additional Context:
- Elasticsearch logs during snapshot and restore processes
- Any configuration settings on Elasticsearch that might be relevant
- Whether you've tested with different versions or settings to isolate the issue
Requests:
- Any known issues regarding snapshot and restore for dense vector indices?
- Suggestions on additional steps I could take to diagnose or fix the issue?
- Any debugging insights or recommendations?