Python APM Agent for Flask doesn't report metrics

[francois.tarlier]

I'm trying to do a simple example in Flask by following the documentation and online tutorials to see how the flask integration works.
I managed to have the span and request working, but I have no metrics reported in Kibana.
also the app.logger doesn't seems to report the logs into Kibana

When I start the Flask server I get the following message :
No handlers could be found for logger "elasticapm.metrics"

Here is my code:

    import logging

    from flask import Flask
    from flask import request
    from flask import render_template

    import elasticapm
    from elasticapm.contrib.flask import ElasticAPM
    from elasticapm.handlers.logging import LoggingHandler

    DEBUG = True

    app = Flask(__name__)
    apm = ElasticAPM(app, service_name="flask-app", logging=logging.INFO)


    @app.route('/')
    def bar():
        """Index."""
        try:
            1 / 0
        except ZeroDivisionError:
            app.logger.error('I cannot math', exc_info=True)


    @app.route('/login', methods=['GET', 'POST'])
    def login():
        """Login."""
        if request.method == 'POST':
            return "post method"
        else:
            return "get method"


    @app.route('/api/<entity>')
    def api(entity):
        """API."""
        limit = int(request.args.get("limit", 0))
        entities = get_entity(entity, limit=limit)
        nb = len(entities)
        app.logger.info("Found {nb} {entity} with limit to {limit}".format(
            nb=nb,
            entity=entity,
            limit=limit))
        return render_template('api.html', 
            entity=entity,
            entities=entities,
            nb=nb)


    @elasticapm.capture_span()
    def get_entity(entity, limit=0):
        """Call to in house DB."""
        entities = db.find(entity,
            limit=limit
        )
        return entities


    if __name__ == '__main__':
        handler = LoggingHandler(client=apm.client)
        handler.setLevel(logging.INFO)
        app.logger.addHandler(handler)
        app.config['ELASTIC_APM'] = {
            'DEBUG': DEBUG
        }
        app.run(host="127.0.0.1", port="8080", debug=DEBUG)

Kibana version:
7.6.0

Elasticsearch version:
7.6.0

APM Server version:
7.6.0

APM Agent language and version:
elastic-apm==5.4.3
Flask==1.1.1

Original install method (e.g. download page, yum, deb, from source, etc.) and version:
docker

Fresh install or upgraded from other version?
local install using docker-compose

[basepi (Colton Myers)]

Welcome to the forum, @francois.tarlier!

I found one issue in your code. You need to do the app.config piece before you set up the apm object, or APM is set up without those config values:

app = Flask(__name__)
app.config["ELASTIC_APM"] = {"DEBUG": DEBUG}
apm = ElasticAPM(app, service_name="flask-app", logging=logging.INFO)

Additionally, did you install psutil? You'll need that to get any of the system metrics.

I took your code, with the one change I showed above, did

pip install flask 'elastic-apm[flask]' psutil
python app.py
curl localhost:8080/login

And saw both transactions and metrics in my local kibana instance.

My guess is that elasticapm.metrics is trying to send the following warning to the logs:

Could not register elasticapm.metrics.sets.cpu.CPUMetricSet metricset: psutil not found. Install it to get system and process metrics

But it's failing because something is odd in the logging. (Probably related to running it in docker -- I didn't tweak the logging at all but I am running it in a local virtualenv.)

Keep me posted if that code change and installing psutil fix the issue for you!

[francois.tarlier]

Thank you, I was missing the psutil like you mentioned. I would recommand to make it more abvious in the documentation or maybe as requirement in elasticapm module.

It is specified in the middle of the documentation in metrics as :
if you do **not** use Linux, you need to install [ psutil ](https://pypi.org/project/psutil/) for this metric set.

It is also needed for OSX.

Thanks a lot for your quick feedback.

F.

[basepi (Colton Myers)]

Thanks for the feedback! In fact, as a result of my investigation of your issue, I added a troubleshooting doc, which includes the note about psutil: https://github.com/elastic/apm-agent-python/pull/756

Interestingly, my coworker pointed out that psutil is only required on non-linux systems. (In your case, docker is probably the culprit.)

Have a great day!