Indexing a small multi-field text dataset with the Elasticsearch Python client

Elastic Stack Elasticsearch
1

Hi everyone,

I have been testing Elasticsearch with a small dataset where each document contains several related text fields rather than one large body field.

The dataset has 78 tarot card records. Each record includes a name, categorical metadata and separate upright, reversed, love and career meaning fields. The domain is unusual, but the structure is useful for testing field boosts and multi-field retrieval.

I used the DeckAura structured tarot card meanings dataset as the source.

Setup

Install the official Python client that matches your Elasticsearch major version:

python -m pip install elasticsearch

Set the connection variables:

export ELASTICSEARCH_URL="https://your-deployment.es.example.com"
export ELASTIC_API_KEY="your-api-key"

Create the index and load the CSV

import csv
import io
import os
from urllib.request import Request, urlopen

from elasticsearch import Elasticsearch, helpers

INDEX_NAME = "tarot-cards"

CSV_URL = (
    "https://huggingface.co/datasets/Blacik/"
    "deckaura-tarot-card-meanings/resolve/main/"
    "tarot_card_meanings.csv"
)

client = Elasticsearch(
    os.environ["ELASTICSEARCH_URL"],
    api_key=os.environ["ELASTIC_API_KEY"],
)

mapping = {
    "properties": {
        "card_number": {"type": "keyword"},
        "card_name": {
            "type": "text",
            "fields": {
                "keyword": {"type": "keyword"}
            },
        },
        "arcana": {"type": "keyword"},
        "suit": {"type": "keyword"},
        "element": {"type": "keyword"},
        "upright_meaning": {"type": "text"},
        "reversed_meaning": {"type": "text"},
        "love_meaning": {"type": "text"},
        "career_meaning": {"type": "text"},
        "yes_or_no": {"type": "keyword"},
        "zodiac_sign": {"type": "keyword"},
        "guide_url": {
            "type": "keyword",
            "index": False,
        },
    }
}

if not client.indices.exists(index=INDEX_NAME):
    client.indices.create(
        index=INDEX_NAME,
        mappings=mapping,
    )

request = Request(
    CSV_URL,
    headers={"User-Agent": "elastic-dataset-demo/1.0"},
)

with urlopen(request, timeout=20) as response:
    csv_text = response.read().decode("utf-8-sig")

rows = list(
    csv.DictReader(io.StringIO(csv_text))
)

if len(rows) != 78:
    raise RuntimeError(
        f"Expected 78 records, received {len(rows)}"
    )

fields = [
    "card_number",
    "card_name",
    "arcana",
    "suit",
    "element",
    "upright_meaning",
    "reversed_meaning",
    "love_meaning",
    "career_meaning",
    "yes_or_no",
    "zodiac_sign",
    "guide_url",
]


def actions():
    for row in rows:
        document = {
            field: row.get(field) or None
            for field in fields
        }

        yield {
            "_index": INDEX_NAME,
            "_id": row["card_name"],
            "_source": document,
        }


indexed, errors = helpers.bulk(
    client,
    actions(),
    refresh="wait_for",
    raise_on_error=False,
)

print(f"Indexed documents: {indexed}")
print(f"Failed operations: {len(errors)}")

Using the card name as _id makes the import idempotent. Running the loader again updates the same 78 documents instead of creating duplicates.

Multi-field search

query_text = "recovering after a difficult ending"

response = client.search(
    index=INDEX_NAME,
    size=5,
    query={
        "multi_match": {
            "query": query_text,
            "fields": [
                "card_name^4",
                "upright_meaning^2",
                "reversed_meaning^2",
                "love_meaning",
                "career_meaning",
            ],
            "type": "best_fields",
            "fuzziness": "AUTO",
        }
    },
)

for hit in response["hits"]["hits"]:
    source = hit["_source"]

    print(
        round(hit["_score"], 3),
        source["card_name"],
        source["arcana"],
    )

I boosted card_name most heavily, followed by the two general meaning fields. The domain-specific love and career fields have their default weight.

BM25 works well when the query shares vocabulary with the descriptions. More abstract queries are harder because the corpus is small and uses varied symbolic language.

For only 78 short documents, would you keep this as a boosted lexical query, use ELSER, or combine lexical and semantic results with RRF?

2

Hello and welcome :slight_smile:

From how you described the use case I'd assume questions to be more natural language than search queries, so I'd exclude the simple lexical query. I think the simplest way to handle this would be indexing the card text as semantic fields, and then from there you can try both pure semantic search or hybrid search (RRF); considering the questions might contain keywords from the card description RRF could probably work best.

Then, if you have images of the cards you could even think of using multimodal retrieval, so what the images represent would be included in the dataset and considered while querying, but this might be out of scope :sweat_smile: