I'm looking to do the same. More specifically, I'm looking to build a DevOps Dashboard that will allow me to surface four key DevOps metrics: change lead time, deployment frequency, mean time to recovery, and change failure rate. I plan to have a breakdown of change lead time so that for each of the applications in my portfolio I can identify the current constraint to allow for the most effective prioritization of improvements.
We are using:
Feature: JIRA
VCS: GitHub
CI: Jenkins
Binary Repo: Nexus
CD: UrbanCode Deploy
QualityMetrics: SonarQube
and many more depending on the application team: SecureAssist, Contrast, AppScan, Nexus IQ, UrbanCode Station, UrbanCode Release, TFS, Dynatrace, Jmeter, RobotFramework, Selenium, etc.
So far I have GitHub web hooks that are sending data to log stash using the http listener and the JSON codec. So far, I'm passing this unchanged into Elastic.
For JIRA, I've found that the webhook sends the object which was most recently changed, which is sometimes the issue, or is sometimes the comment. I wanted to always have the issue and the issue changelog, so I've instructed JIRA to not send the JSON payload and instead send the issue id in the query string. I then use the log stash-filter-rest to call back to JIRA using it's REST api to get the associated issue, along with the changelog. I've set the document id to be the id of the JIRA issue which means that I'm storing at most one copy in elastic for each issue. A new update will overwrite the old one. I may revisit this decision in future. I've also set the timestamp for the event as the last updated timestamp from JIRA. I may revisit this decision in future as well.
I'm planning to use JIRA and GitHub smart commits to tie code checkins to a JIRA issue. I have this configured, but my teams are not using it in practice yet. Initially it will be voluntary to include a reference to a JIRA issue in a commit/push. I may find that I need to make it required somewhere down the line.
I plan to start my primary change lead time clock when an issue is first prioritized to be worked on. I'll interpret this as when the issue is added to a sprint, or when the status changes from a status in the status category To Do to a status in the category of In Progress. I plan to glean this information from the changelog associated with the issue.
So each GitHub checkin will be associated with one or more JIRA issues. We also have web hooks that queue build jobs in Jenkins, and so Jenkins also ends up with a copy of the GitHub commit messages which now ties together the trio of Feature, VCS, and CI.
Now I'll need to actually track the progression of the package that was created from the Jenkins build through the Dev, QA, and Production environments. Jenkins' fingerprinting idea has inspired my plan to compute a hash of the package as it is deployed into each of the environments, and tie that back to the build along with any quality statuses or verification attributes that the package picks up as it progresses through each phase of validation.
I currently have arrived at the impression that I would be best suited to compute the data points that I want to report upon in aggregate, for each of the events at the time of ingestion. As an example if I want to report on the elapsed time from when a JIRA issue was created to when it was first in progress or added to a sprint then I'm planning to store this calculated value in the event by computing it in a log stash filter. I think that if I do this, then my work in Kibana is going to be much more straight forward. Getting started with this will be my next step.
Next after this I'm going to need the Jenkins logs. In my environment there is an enterprise instance of Jenkins and I'm testing out the feasibility of moving to a dedicated instance where I would have more control over the configuration. This may take a while.
==========================================
The DevOps handbook, Accelerate, Beyond the Phoenix Project, and DORA all speak of the power of metrics and data driven insights. I'm fully bought into the idea that my teams would prioritize improvements more effectively if empowered with a centralized view aggregated from this collection of best of bread tools. At this time my best idea to tackle this is to build it ourselves using Elastic. My back pocket idea that I haven't looked into to deeply is to partner with Tasktop.
I'm consumed every Elastic video that I can find that has any remote mention of DevOps, and while there are many, none @Shirm are trying to do what you and I are. I'm encouraged to have found someone who is thinking along the same lines. Please elaborate on your plans and let me know how you would improve upon what I've been considering and what I've gotten stated with.