How to view the backend log of the Python class `Elasticsearch`?

Elastic Stack Elasticsearch
1

We are using an instance of the Python class Elasticsearch, e.g., by es = Elasticsearch(hosts="http://test-elastic-host:9200").

For example, when calling the search() method, we need to know what exactly the request is, sending to the Elasticsearch instance on the backend.

Our Question:

So, we wonder if there is logging for the backend requests, like the Django DB backend logging of SQL statements sending to the database.

Sample source code of using the Python class Elasticsearch:

def search_transactions(product_id, search_term, from_, size):
    es = Elasticsearch(hosts="http://test-elastic-host:9200")
    es_query = {
        "match": {
            "customer.email": search_term
        }
    }

    es_response_count = es.count(
        index="my-index-000001",
        query=es_query)
    es_count = es_response_count["count"]

    es_response_search = es.search(
        index="my-index-000001",
        query=es_query,
        from_=from_,
        size=size
    )
    es_hits = es_response_search['hits']['hits']

    es.transport.close()

    return es_count, es_hits
2

Hi, yes, the Elasticsearch Python client supports logging of requests made to the Elasticsearch backend. You can enable logging by configuring the Python logging module. Here's an example of how to do it:

import logging
from elasticsearch import Elasticsearch

# Enable Elasticsearch logging
es_logger = logging.getLogger('elasticsearch')
es_logger.setLevel(logging.INFO)

# Create console handler
console_handler = logging.StreamHandler()
console_handler.setLevel(logging.INFO)

# Create formatter
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')

# Add formatter to console handler
console_handler.setFormatter(formatter)

# Add console handler to logger
es_logger.addHandler(console_handler)

# Now, when you make a request, it will be logged
es = Elasticsearch(hosts="http://test-elastic-host:9200")
response = es.search(index="my-index-000001", body={"query": {"match_all": {}}})

In this example, the logger is set to log at the INFO level, which will log all the requests made to the Elasticsearch backend. You can adjust the logging level as per your needs. The logs will be printed to the console because of the StreamHandler. If you want to log to a file, you can use a FileHandler instead.

Remember to close the logger and the Elasticsearch connection when you're done:

# Close the connection and logger when done
es.transport.close()
for handler in es_logger.handlers:
    handler.close()
    es_logger.removeHandler(handler)

This will give you a detailed log of all the requests made to the Elasticsearch backend, similar to Django's SQL statement logging.

Side note: opsgpt.io helped me with part of this answer.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.