Hey @alex-treebeard . Thanks for the question for interest in our work!
Most of the tooling that we use for the bots is available in our pipeline library for Jenkins, which is open-source and can be seen here:
GitHub - elastic/apm-pipeline-library: Jenkins pipeline shared library for the APM project.
The flaky test analyzer is not currently a part of that library but what it does is more or less what you might imagine. It looks through branch test results and applies heuristics to try to find tests which fail intermittently. We've also been experimenting more recently with more advanced forms of analysis which attempts to find and categorize build failures by type (test-related, CI-related, infrastructure-related etc) by querying a mix of the Jenkins API and the test results stored in Elasticsearch. If I get a bit more time, I'll try to put together a blog post or something on how that all works.
If you have specific questions in the meantime, I'm happy to answer them. Cheers and good luck with your project! I think there's a lot of interest in something like what you're building and I'll be very interested to see what you come up with.
