Dec 4th, 2024: [EN] Does the Elasticsearch Python client work with free-threading Python?

Quentin_Pradet · December 4, 2024, 8:00am

In this post, we'll run some experiments to see if the Python Elasticsearch client is compatible with the new Python 3.13 free-threading builds, where the GIL was removed.

Introduction

But first, what is the GIL? The Global Interpreter Lock (GIL) is a mutex that protects access to Python objects, preventing multiple threads from executing Python bytecode at once. This is not always a problem in practice.

Scientific programming can use libraries like NumPy that don't hold the GIL.
Some programs are not CPU-bound but I/O-bound. For example, if your code is issuing expensive requests to Elasticsearch and doesn't make expensive computations on the results, it can use multiple threads effectively. Indeed, even if only one thread is executing, it won't block the others that will be waiting for I/O and thus won't block the GIL. (This is also the scenario where async/await shines in Python.)

However, it has been a goal for decades to lift this limitation and allow true multi-threading programming. Thanks to incredible work from Sam Gross, it is now possible! This work was initially called nogil, but is now called free-threading. While existing pure Python code still works the same with the resulting builds (albeit currently slower for single-threaded code), all code compiled from other languages like C or Rust needs refactoring. In the past, such a backwards incompatible change would have been enough justification to release Python 4. However, the Python 3 migration resulted in a language split for more than 10 years, and the resulting pain is still fresh in many minds. As a result, the current plan is a gradual rollout:

As part of phase 1 (the current phase), Python 3.13 provides experimental free-threading builds, and it's up to each library and application to test their compatibility.
In phase 2, those builds won't be called "experimental" anymore.
In phase 3 the standard Python build will include free-threading support.

The Elasticsearch Python client is pure Python code without much threading involved or specific reliance on the garbage collector, so it should work just as well with free-threading builds. However, it does have optional dependencies that are affected, such as aiohttp or orjson.

We're going to test those various parts and see if they work. Benchmarking will be left as an exercise to the reader!

Using free-threading Python

There are various ways to install free-threading Python builds. We'll use Astral's uv package manager which allows specifiying free-threading builds with --python 3.13t. Astral contributed free-threading builds to python-build-standalone, and uv will use them if needed:

$ uv run --python 3.13t python
Using CPython 3.13.0
Creating virtual environment at: .venv
Installed 4 packages in 16ms
Python 3.13.0 experimental free-threading build (main, Oct 16 2024, 08:24:33)
[Clang 18.1.8 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>

However, if you already have a free-threading interpreter installed, uv will use it instead of python-build-standalone. For example, if you want to use the build provided by Homebrew on macOS (installed with brew install python-freethreading), you will get the following output instead:

$ uv run --python 3.13t python
Using CPython 3.13.0 interpreter at:
/opt/homebrew/opt/python-freethreading/bin/python3.13t
Creating virtual environment at: .venv
Installed 4 packages in 4ms
Python 3.13.0 experimental free-threading build (main, Oct  7 2024, 05:02:14)
[Clang 16.0.0 (clang-1600.0.26.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>

And since uv also supports the Inline scripting metadata standard, we'll provide self-contained snippets like this:

# /// script
# requires-python = ">=3.13"
# dependencies = [
#     "numpy",
# ]
# ///
import numpy as np

c = np.arange(24).reshape(2, 3, 4)

You can run them without having to worry about virtual environment or installing dependencies manually:

$ uv run --python 3.13t example.py
Reading inline script metadata from `example.py`
[[[ 0  1  2  3]
  [ 4  5  6  7]
  [ 8  9 10 11]]

 [[12 13 14 15]
  [16 17 18 19]
  [20 21 22 23]]]

Using Elasticsearch

Thanks to the start-local script, Elasticsearch is similarly easy to run:

$ curl -fsSL https://elastic.co/start-local | sh

  ______ _           _   _
 |  ____| |         | | (_)
 | |__  | | __ _ ___| |_ _  ___
 |  __| | |/ _` / __| __| |/ __|
 | |____| | (_| \__ \ |_| | (__
 |______|_|\__,_|___/\__|_|\___|
-------------------------------------------------
🚀 Run Elasticsearch and Kibana for local testing
-------------------------------------------------

ℹ️  Do not use this script in a production environment

⌛️ Setting up Elasticsearch and Kibana v8.16.0...

- Generated random passwords
- Created the elastic-start-local folder containing the files:
  - .env, with settings
  - docker-compose.yml, for Docker services
  - start/stop/uninstall commands
- Running docker compose up --wait

🎉 Congrats, Elasticsearch and Kibana are installed and running in Docker!

🌐 Open your browser at http://localhost:5601
🔌 Elasticsearch API endpoint: http://localhost:9200

Let's test it:

# /// script
# requires-python = ">=3.13"
# dependencies = [
#     "elasticsearch",
# ]
# ///
import os
import sys

from elasticsearch import Elasticsearch

print(sys.version)
client = Elasticsearch(
    "http://localhost:9200", api_key=os.environ["ES_LOCAL_API_KEY"]
)
print(client.info()["tagline"])

While start-local does not use HTTPS, it does set up authentication. The relevant secrets are stored in the elastic-start-local/.env file so we can source it and pass ES_LOCAL_API_KEY as an environment variable:

$ source elastic-start-local/.env
$ ES_LOCAL_API_KEY=$ES_LOCAL_API_KEY uv run --python 3.13t ex1.py
Reading inline script metadata from `ex1.py`
3.13.0 experimental free-threading build (main, Oct 16 2024, 08:24:33)
[Clang 18.1.8 ]
You Know, for Search

Great! A simple query worked as expected. Now, let's test other areas of the Python client.

Bulk helper

The only place where we explicitly use threads in the Python client is in the parallel_bulk helper. Let's index the books.csv dataset and make a query to see if that worked.

# /// script
# requires-python = ">=3.13"
# dependencies = [
#     "elasticsearch",
# ]
# ///
import csv
import os
import sys
import time

from elasticsearch import Elasticsearch, helpers

client = Elasticsearch(
    "http://localhost:9200", api_key=os.environ["ES_LOCAL_API_KEY"]
)

mappings = {
    "properties": {
        "Title": {"type": "text"},
        "Description": {"type": "text"},
        "Author": {"type": "text"},
        "Year": {"type": "date", "format": "yyyy"},
        "Published": {"type": "keyword"},
        "Rating": {"type": "scaled_float", "scaling_factor": 100},
    }
}

client.options(ignore_status=[404]).indices.delete(index="books")
client.indices.create(index="books", mappings=mappings)
print("Created index")


def generate_docs():
    with open("books.csv", newline="") as csvfile:
        reader = csv.DictReader(csvfile, delimiter=";")
        for row in reader:
            yield {"_index": "books", **row}


start = time.perf_counter()
n, errors = helpers.bulk(client, generate_docs())
end = time.perf_counter()
print(f"Indexed {n} books in {end - start:.1f} seconds.")

client.indices.refresh(index="books")

print("Searching for Stephen King:")
resp = client.search(
    index="books", query={"match": {"Author": "Stephen King"}}
)
for hit in resp.body["hits"]["hits"]:
    book = hit["_source"]
    description = f'{book["Author"]} - {book["Title"]} ({book["Year"]})'
    rating = f'{book["Ratings"]} stars'
    print(f"  {description}: {rating}")

The output of the script showed that we indeed indexed all 82k books in less than 2 seconds! This is about 2x faster than with the standard bulk helper.

$ ES_LOCAL_API_KEY=$ES_LOCAL_API_KEY uv run --python 3.13t ex2.py
Reading inline script metadata from `ex2.py`
Created index
Indexed 81828 books in 1.6 seconds.
Searching for Stephen King:
  Stephen King - THE ELEMENTS OF STYLE (2013): 5.00 stars
  Stephen King - Star (Thorndike Core) (2021): 3.11 stars
  Stephen King - Hearts in Atlantis (2017): 4.08 stars
  Stephen King - Misery (Spanish Edition) (2016): 4.43 stars
  Stephen King - The Dead Zone (2016): 4.40 stars
  Stephen King - Another World (Thorndike Core) (2021): 3.78 stars
  Stephen King - FROM A BUICK 8 (True first edition) (2017): 3.25 stars
  Stephen King - Road Work (2017): 4.29 stars
  Stephen King - Icon (Thorndike Core) (2021): 4.00 stars
  Stephen King - Misery (2016): 4.43 stars

aiohttp

The Elasticsearch Python client supports asyncio with two HTTP clients, aiohttp and httpx, the default being aiohttp. While aiohttp does not officially support free-threading builds yet (and indeed currently fails to compile), it's possible to use it in pure-Python mode by setting AIOHTTP_NO_EXTENSIONS=1. It's going to be slower but will work with free-threading builds.

In terms of testing, there's not much to test. The asyncio event loop is limited to a single thread already. Let's reuse the example from earlier but using asyncio instead:

# /// script
# requires-python = ">=3.13"
# dependencies = [
#     "elasticsearch[async]",
# ]
# ///
import asyncio
import os
import sys

from elasticsearch import AsyncElasticsearch

print(sys.version)


async def main():
    async with AsyncElasticsearch(
        "http://localhost:9200", api_key=os.environ["ES_LOCAL_API_KEY"]
    ) as client:
        info = await client.info()
        print(info["tagline"])


asyncio.run(main())

Since uv run installs dependencies on the fly, we need to define AIOHTTP_NO_EXTENSIONS to run. And indeed, the script behaves as expected:

$ export AIOHTTP_NO_EXTENSIONS=1
$ export ES_LOCAL_API_KEY=$ES_LOCAL_API_KEY
$ uv run --python 3.13t ex3.py
Reading inline script metadata from `ex3.py`
3.13.0 experimental free-threading build (main, Oct 16 2024, 08:24:33
[Clang 18.1.8 ]
You Know, for Search

Serialization and deserialization

The Elasticsearch Python client supports multiple libraries to serialize or deserialize data. They often use native code for performance reasons, and those libraries need to be adapted to work with free-threading builds.

orjson allows fast serialization/deserialization of JSON but does not support free-threading builds yet and does not even compile.

PyArrow 18+ and Pandas 2.2.3+ support free-threading builds. Let's reuse the books index by making an ES|QL query instead:

# /// script
# requires-python = ">=3.13"
# dependencies = [
#     "elasticsearch",
#     "pandas",
#     "pyarrow",
# ]
# ///
import csv
import os
import sys
import time

import pandas as pd
from elasticsearch import Elasticsearch, helpers

client = Elasticsearch(
    "http://localhost:9200", api_key=os.environ["ES_LOCAL_API_KEY"]
)

print("Searching for Stephen King:")
resp = client.esql.query(
    query="""
    FROM books
    | WHERE Author == "Stephen King"
    | SORT Rating DESC
    | LIMIT 10
    """,
    format="arrow",
)
df = resp.to_pandas(types_mapper=pd.ArrowDtype)
print(df)

This outputs the following:

$ PYTHON_GIL=0 ES_LOCAL_API_KEY=$ES_LOCAL_API_KEY uv run --python 3.13t ex4.py
Reading inline script metadata from `ex4.py`
Searching for Stephen King:
         Author  ...                                Title                 Year
0  Stephen King  ...       Another World (Thorndike Core)  2021-01-01 00:00:00
1  Stephen King  ...  FROM A BUICK 8 (True first edition)  2017-01-01 00:00:00
2  Stephen King  ...                   Hearts in Atlantis  2017-01-01 00:00:00
3  Stephen King  ...             Misery (Spanish Edition)  2016-01-01 00:00:00
4  Stephen King  ...       The Dark Tower: The Gunslinger  2017-01-01 00:00:00
5  Stephen King  ...                        The Dead Zone  2016-01-01 00:00:00
6  Stephen King  ...           NIGHTMARES AND DREAMSCAPES  2017-01-01 00:00:00
7  Stephen King  ...                    How writers write  2002-01-01 00:00:00
8  Stephen King  ...                THE ELEMENTS OF STYLE  2013-01-01 00:00:00
9  Stephen King  ...                            Road Work  2017-01-01 00:00:00

Note that I had to set PYTHON_GIL=0 to disable the following warning which I don't think should not be emitted as those libraries do support free-threading builds. Maybe this will be fixed in a future version.

<frozen importlib._bootstrap>:488: RuntimeWarning: The global interpreter lock (GIL) has been enabled to load module 'pandas._libs.pandas_parser', which has not declared that it can run safely without the GIL. To override this behavior and keep the GIL disabled (at your own risk), run with PYTHON_GIL=0 or -Xgil=0.

Conclusion

In conclusion, the free-threading builds work surprisingly well! Many important libraries already support free-threading. While there are still some unsupported libraries like orjson or Polars, they are the exceptions, not the rule. The future is bright for free-threading, and I can see those builds getting out of experimental status quickly. (But until this happens, I would advise against using them in production.)

If you want to learn more about free-threading, https://py-free-threading.github.io/ is an excellent resource, and the More Resources page in particular links to useful learning material.

And to answer my initial question: yes, the Python Elasticsearch client works great with free-threading!

Topic		Replies	Views
Dec 4th, 2024: [FR] Est-ce que le client Python Elasticsearch fonctionne sans le GIL? Advent Calendar	0	19	December 4, 2024
Threading and the Elasticsearch Context in Python Elasticsearch language-clients	4	1182	November 10, 2022
ElasticSearch multi-threading and the Java EE specifications Elasticsearch	10	1402	July 6, 2017
Dec 17th, 2021: [EN] Getting Started with the Elasticsearch Python Client Advent Calendar	1	1576	January 14, 2022
Is elasticsearch client is thread safe? Elasticsearch	3	2510	November 12, 2018