Calculate MTBF/MTTF Mean time between/to failure for a visualization

topel · June 7, 2018, 1:13pm

Hi there,

I am struggling to find the way or rather a way to visualize the MTBF of my Jenkins Pipelines/Jobs.

What I have is the following:

A job index, where one document holds the information about what builds belong to this job (as an Array of urls)
This also contains information about what the last successful build and the last unsuccessful build are (urls)
A build index that has the detailed information for each build with a starting timestamp, duration and of course a result field as well as a reference to what pipeline it belogs to (url)

This is what I have but it is still fully adjustable if you tell me, that there is a better approach.

I tried using timelion but I am stuck at the point where I have to decide where my time difference (the tbf part of mtbf) comes from.

As a side-note: The mtbf for me only makes sense for pipeline so I have to filter the corresponding builds at some point to only get the ones I really need.

I am aware I have only given minimal information at the point, I'll gladly provide any additional information if needed.

@Bargs answered a similar question back in 2017 but I think it is not fully applicable here.

Any advice?

Cheers and thanks for the help,
Tobias

chrisronline · June 7, 2018, 6:40pm

Hi @topel,

A couple things:

Can you provide me with some sample data and the mappings for your indices?
Can you provide a link to the related issue back in 2017?

Thanks,
Chris

topel · June 8, 2018, 8:22am

Hi @chrisronline,

I just double checked the issue was this one and I seem to have recollected it wrong - it only vaguely touches the topic.

Sample job data:

ES job index entry

 {
    "_index": "test-jobs-old",
    "_type": "doc",
    "_id": "yL_j1WMBkhQDVC7mqY67",
    "_score": 1,
    "_source": {
      "displayName": "GroovyTester_Pipeline",
      "@timestamp": "2018-06-06T16:17:25.649Z",
      "description": "",
      "lastSuccessfulBuild": {
        "url": "https://bla.com/job/GroovyTester_Pipeline/23/",
        "class": "org.jenkinsci.plugins.workflow.job.WorkflowRun",
        "number": 23
      },
      "firstBuild": {
        "url": "https://bla.com/job/GroovyTester_Pipeline/1/",
        "class": "org.jenkinsci.plugins.workflow.job.WorkflowRun",
        "number": 1
      },
      "concurrentBuild": true,
      "resumeBlocked": false,
      "keepDependencies": false,
      "beat": {
        "name": "dashboard",
        "hostname": "dashboard",
        "version": "6.2.4"
      },
      "healthReport": {
        "buildStability": {
          "remark": "4 out of the last 5 builds failed.",
          "score": 20
        }
      },
      "fullName": "GroovyTester_Pipeline",
      "nextBuildNumber": 24,
      "inQueue": false,
      "parameterDefinitions": {},
      "url": "https://bla.com/job/GroovyTester_Pipeline/",
      "lastStableBuild": {
        "url": "https://bla.com/job/GroovyTester_Pipeline/23/",
        "class": "org.jenkinsci.plugins.workflow.job.WorkflowRun",
        "number": 23
      },
      "fullDisplayName": "GroovyTester_Pipeline",
      "lastUnsuccessfulBuild": {
        "url": "https://bla.com/job/GroovyTester_Pipeline/22/",
        "class": "org.jenkinsci.plugins.workflow.job.WorkflowRun",
        "number": 22
      },
      "name": "GroovyTester_Pipeline",
      "lastCompletedBuild": {
        "url": "https://bla.com/job/GroovyTester_Pipeline/23/",
        "class": "org.jenkinsci.plugins.workflow.job.WorkflowRun",
        "number": 23
      },
      "host": "dashboard",
      "buildable": true,
      "offset": 1394532,
      "prospector": {
        "type": "log"
      },
      "color": "blue",
      "actions": [
        "hudson.plugins.jobConfigHistory.JobConfigHistoryProjectAction",
        "com.cloudbees.plugins.credentials.ViewCredentialsAction",
        "hudson.plugins.buildblocker.BuildBlockerProperty",
        "com.sonyericsson.rebuild.RebuildSettings",
        "com.synopsys.arc.jenkinsci.plugins.jobrestrictions.jobs.JobRestrictionProperty",
        "org.jenkinsci.plugins.workflow.job.properties.PipelineTriggersJobProperty"
      ],
      "lastFailedBuild": {
        "url": "https://bla.com/job/GroovyTester_Pipeline/22/",
        "class": "org.jenkinsci.plugins.workflow.job.WorkflowRun",
        "number": 22
      },
      "tags": [
        "json",
        "jenkins",
        "jobs",
        "init",
        "beats_input_raw_event"
      ],
      "builds": [
        "https://bla.com/job/GroovyTester_Pipeline/23/",
        "https://bla.com/job/GroovyTester_Pipeline/22/",
        "https://bla.com/job/GroovyTester_Pipeline/21/",
        "https://bla.com/job/GroovyTester_Pipeline/20/",
        "https://bla.com/job/GroovyTester_Pipeline/19/",
        "https://bla.com/job/GroovyTester_Pipeline/18/",
        "https://bla.com/job/GroovyTester_Pipeline/17/",
        "https://bla.com/job/GroovyTester_Pipeline/16/",
        "https://bla.com/job/GroovyTester_Pipeline/15/",
        "https://bla.com/job/GroovyTester_Pipeline/14/",
        "https://bla.com/job/GroovyTester_Pipeline/13/",
        "https://bla.com/job/GroovyTester_Pipeline/12/",
        "https://bla.com/job/GroovyTester_Pipeline/11/",
        "https://bla.com/job/GroovyTester_Pipeline/10/",
        "https://bla.com/job/GroovyTester_Pipeline/9/",
        "https://bla.com/job/GroovyTester_Pipeline/8/",
        "https://bla.com/job/GroovyTester_Pipeline/7/",
        "https://bla.com/job/GroovyTester_Pipeline/6/",
        "https://bla.com/job/GroovyTester_Pipeline/5/",
        "https://bla.com/job/GroovyTester_Pipeline/4/",
        "https://bla.com/job/GroovyTester_Pipeline/3/",
        "https://bla.com/job/GroovyTester_Pipeline/2/",
        "https://bla.com/job/GroovyTester_Pipeline/1/"
      ],
      "lastBuild": {
        "url": "https://bla.com/job/GroovyTester_Pipeline/23/",
        "class": "org.jenkinsci.plugins.workflow.job.WorkflowRun",
        "number": 23
      },
      "source": "/srv/jenkins_data/init_jobs_log_2018-06-06_163617.json",
      "@version": "1",
      "class": "org.jenkinsci.plugins.workflow.job.WorkflowJob",
      "property": []
    }
  }

topel · June 8, 2018, 10:29am

Sample build data:

ES build index entry

{
    "_index": "test-builds-old",
    "_type": "doc",
    "_id": "mL_j1WMBkhQDVC7m4sHz",
    "_score": 1,
    "_source": {
      "displayName": "#31-pipeline-name",
      "@timestamp": "2018-03-16T13:12:32.775Z", 
      "changeSetMercurial": [],
      "result": "SUCCESS",             < ----------------------- 
      "keepLog": false,
      "building": false,
      "nextBuild": {
        "number": 32,
        "url": "https:/bla.com/job/package-promotion-pipeline/32/"
      },
      "causeAction": {
        "rebuildTrigger": {
          "upstreamBuild": 30,
          "upstreamUrl": "job/package-promotion-pipeline/",
          "shortDescription": "Rebuilds build #30",
          "upstreamProject": "package-promotion-pipeline"
        },
        "userTrigger": {
          "userId": "johndoe",
          "shortDescription": "Started by user Doe, John",
          "userName": "Doe, John"
        }
      },
      "beat": {
        "name": "dashboard",
        "hostname": "dashboard",
        "version": "6.2.4"
      },
      "number": 31,
      "id": "31",
      "previousBuild": {
        "number": 30,
        "url": "https://bla.com/job/package-promotion-pipeline/30/"
      },
      "pipelineUrl": "https://bla.com/job/package-promotion-pipeline/",
      "fullDisplayName": "package-promotion-pipeline #31-pipeline-name",
      "url": "https://bla.com/job/package-promotion-pipeline/31/",
      "queueId": 68074,
      "culprits": [],
      "mercurialTags": [
        ...
      ],
      "buildData": [
        ...
      ],
      "duration": 12110,
      "host": "dashboard",
      "estimatedDuration": 16512,
      "offset": 68405728,
      "prospector": {
        "type": "log"
      },
      "actions": [
        "hudson.plugins.git.GitTagAction",
        "org.jenkinsci.plugins.workflow.cps.EnvActionImpl",
        "org.jenkinsci.plugins.workflow.job.views.FlowGraphAction"
      ],
      "changeSetGit": [],
      "tags": [
        "json",
        "jenkins",
        "builds",
        "init",
        "beats_input_raw_event"
      ],
      "@version": "1",
      "source": "/srv/jenkins_data/init_builds_log_2018-06-06_163617.json",
      "class": "org.jenkinsci.plugins.workflow.job.WorkflowRun",
      "timestamp": 1521202340665 
    }
  }

I marked the build result with an arrow. What I now want is to filter for one pipeline and then do maths on the time it took between the different results. So for example what the average time is between one build with result failure and the next one.

Hope this clarifies things and you are now able to help me.

Cheers,
Tobias

topel · June 14, 2018, 8:10am

@chrisronline bump

system · July 12, 2018, 8:10am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Calculate MTTR for jenkins builds Elasticsearch transforms	3	325	April 3, 2023
Create visualize -> Success rate based on time Kibana	4	570	September 9, 2022
Calculating MTTR(Mean Time to Recovery) Kibana	2	820	December 17, 2020
Kibana fetch data and visualization Kibana	7	1083	July 31, 2018
Jenkins Logstash plugin to show current state of all jobs in one Kibana graph Kibana	4	1500	July 18, 2018

Calculate MTBF/MTTF Mean time between/to failure for a visualization

Related topics