# Dec 3rd 2021: \[en\] What's new in the Elasticsearch Python client 8.0.0

**URL:** <https://discuss.elastic.co/t/dec-3rd-2021-en-whats-new-in-the-elasticsearch-python-client-8-0-0/289203>\
**Category:** Advent Calendar\
**Created:** [December 3, 2021, 8:00am UTC](https://discuss.elastic.co/t/dec-3rd-2021-en-whats-new-in-the-elasticsearch-python-client-8-0-0/289203 "2021-12-03T08:00:33Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![sethmlarson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sethmlarson/32/93342_2.png) [@sethmlarson](https://discuss.elastic.co/u/sethmlarson)\
**Post date:** [December 3, 2021, 8:00am UTC](https://discuss.elastic.co/t/dec-3rd-2021-en-whats-new-in-the-elasticsearch-python-client-8-0-0/289203/1 "2021-12-03T08:00:33Z")

</div>

There's an [8.0.0 pre-release out for the Elasticsearch client](https://github.com/elastic/elasticsearch-py/releases/tag/v8.0.0a1), fresh off the press! You can give it a try yourself or skip to the "What's new" section to read about all the new features coming in 8.0.0.

# Getting started

Start by installing the pre-release with `pip` with the `--pre` flag to opt-in to pre-releases:

```auto
$ python -m pip install --pre 'elasticsearch>=8'

```

Then you can [start an Elasticsearch 8.0.0 preview cluster and connect](https://www.elastic.co/guide/en/elasticsearch/reference/8.0/docker.html). Note that by default security will be enabled meaning you need to capture the password of the `elastic` user. If your cluster generated a certificate bundle you should copy the `http_ca.crt` file and use `ca_certs` to specify its location.

```auto
from elasticsearch import Elasticsearch

client = Elasticsearch(
  "https://localhost:9200",
  basic_auth=("elastic", "<password>"),
  ca_certs="/path/to/http_ca.cert"
)

```

If you're unable to get this working and want to do a quick test you can use `verify_certs=False`, but don't use this setting for a production cluster!

## Find an issue? Report it to us!

At the time of writing the released version is only an 8.0.0 pre-release, that means there's time for us to fix bugs! If you happen to find something [please open an issue on our repository](https://github.com/elastic/elasticsearch-py/issues).

# What's new in the 8.0.0 pre-release?

Now that you've setup your cluster and client, what sorts of features are new in the pre-release?

### Richer responses

In previous versions of the client, responses were only raw deserialized JSON. Because objects like `dict` and `list` were used this meant you couldn't access fine-grained details about the transport layer like HTTP status, headers, and more. In 8.0.0 you can access `resp.meta` for lots of details about the response:

```python
resp = client.info()

>>> resp.meta.status # HTTP status code
200
>>> resp.meta.headers # HTTP headers
{'Content-Encoding': 'gzip',
 'Content-Length': '348',
 'Content-Type': 'application/json', ...}
>>> resp.meta.http_version # HTTP version
'1.1'
>>> resp.meta.node # Which node serviced the request?
NodeConfig(scheme='https', host='6d1f...a5af.us-west2.gcp.elastic-cloud.com', port=443, ...)

```

More fields and data will be added to this `meta` object in the future as they become available to the transport library.

Because responses are no longer simple `dict` instances if you really need access to the underlying `dict` you can use the `.raw` property. Keep in mind that typical access patterns like indexing, calling methods, and special Python functions like `len`, `iter`, etc will all work identically:

```python
# The response object isn't a dictionary, it's an object:
>>> resp
ObjectApiResponse({'name': 'instance-0000000000', ..., 'tagline': 'You Know, for Search'})

# You can access the response via indexing like in 7.x:
>>> resp["tagline"]
'You Know, for Search'

# Methods of the underlying type are forwarded similarly:
>>> resp.keys()
dict_keys(['name', 'cluster_name', 'cluster_uuid', 'version', 'tagline'])

# If you need the underlying 'dict' you can use .raw:
>>> resp.raw
{'name': 'instance-0000000000', ..., 'tagline': 'You Know, for Search'}

```

## Body field parameters

In previous versions of the client all information encoded into the HTTP body was provided via an opaque `body` parameter. This meant in some of the most complex parts of the Elasticsearch API you'd get no help with auto-complete or type hints. Now in 8.0.0 thanks to the [Elasticsearch specification project](https://github.com/elastic/elasticsearch-specification) we now have a complete API specification of all parts the Elasticsearch API, including HTTP bodies!

This means we can provide top-level parameters for bodies in addition to path and query parameters. You don't have to figure out where parameters are serialized, just use the parameters and let the client figure that out for you:

```python
client.ingest.put_pipeline(
  id="pipeline-id",
  master_timeout="5m",
  description="My ingest pipeline",
  processors=[
    {"set": {"field": "ingested_at", "value": "{{_ingest.timestamp}}"}}
  ],
  on_failure=[
    {"set": {"field": "_index", "value": "failed-docs"}}
  ]
)

```

Here's the same API request in 7.x, notice you have to know which parameters go into `body` and which don't:

```python
client.ingest.put_pipeline(
  id="pipeline-id",
  master_timeout="5m",
  body={
    "description": "My ingest pipeline"
    "processors": [
      {"set": {"field": "ingested_at", "value": "{{_ingest.timestamp}}"}}
    ],
    "on_failure": [
      {"set": {"field": "_index", "value": "failed-docs"}}
    ]
  }
)

```

## Per-request options

To change a parameter for a single request you can modify a small subset of transport options like `api_key`, `request_timeout`, `opaque_id` (and a few more) directly within the API method:

```python
# This usage is deprecated, don't use in v8.0+
client.search(..., api_key=("id", "api-key"))

```

This worked for most APIs, however there are conflicts with certain APIs that have query parameters with the same name as the transport parameters (like the `sql.query` API accepting `request_timeout`). For these situations is the parameter used for the transport or for the API?

We make the distinction clear with the new `Elasticsearch.options()` method which can be used to build new client instances using the existing transport instance with slightly different options:

```python
client.options(api_key=("id", "api-key")).search(...)

```

Because a new client instance is returned from the `options()` method you can also build contextualized clients that have different authentication and use them over and over instead of having to pass the authentication into every API call:

```python
auth_client = not_auth_client.options(basic_auth=("username", "password"))

# This `auth_client` instance has the username:password privileges
resp = auth_client.indices.create(...)

```

## Explicit client configuration

Previously when instantiating an `Elasticsearch` client instance the default URL would be `scheme="http"`, `host="localhost"`, and `port=9200`, however the method that resolved each parameter was very complicated and didn't make a lot of sense in many situations (should a scheme of `https` give a default port of `443`, `9200`, `9243`, or `9443`? Elastic Cloud used to use one of these ports).

There can be even more confusion with 8.0.0 enabling HTTPS by default ( **which is a huge win for user security!** ) This means the 7.x default value (`http://localhost:9200`) doesn't work with a default 8.0 Elasticsearch server instance, instead should be `https://localhost:9200`.

Because of this starting in 8.0.0 the client must be explicitly configured with either a URL including a scheme and port (like `https://localhost:9200`) or via an Elastic Cloud ID (`cloud_id="deployment:..."`). This change ensures we're never using bad or confusing defaults.

## Keyword-only arguments

In v7.15.0 of the client using positional arguments would raise a `DeprecationWarning`. Starting in 8.0.0 all parameters on API methods are keyword-only and passing positional arguments to APIs will raise a `TypeError`.

```python
# Keyword arguments only!
client.search(index="target-index")

# This will raise a TypeError:
client.search("target-index")

```

Requiring keyword arguments makes your code resilient to additional API parameters being added or changed and makes your code more readable: **Double win!**

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 31, 2021, 8:01am UTC](https://discuss.elastic.co/t/dec-3rd-2021-en-whats-new-in-the-elasticsearch-python-client-8-0-0/289203/2 "2021-12-31T08:01:30Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.