Skip to content

Latest commit

 

History

History
149 lines (110 loc) · 8.32 KB

code_development.md

File metadata and controls

149 lines (110 loc) · 8.32 KB
Raw

Focus Area: Code Development

Scope: Aspects related to how the source code changes over time, and the mechanisms that the project has to perform and control those changes.

Goals

  • Activity: Learning about activity involved in developing (e.g. adding/changing) code
  • Efficiency: Learning how effective new code is merged into the code base
  • Quality: Learning about the processes to improve/review quality that are used (for example: testing, code review)

Observations:

  • Usually these goals, which in general correspond to the evaluation of some processes, are considered over a certain time period. Therefore, questions and metrics will also refer, usually, to time periods.

  • Since the focus area is code development, goals (and therefore relevant questions and metrics) are specific for code development. Therefore, any metric related to these goals should be considered as filtering only data relevant to code development (for example, changes only to source code files). In any case, very likely these same goals could be applied to other artifacts (such as documentation) when it is developed in a similar way to the source code.

  • For Quality, we assume that there are measurable processes to improve/review quality (testing, code review, etc.). If they don't exist, this goal cannot be satisfied.

Questions & metrics

[Work in progress]

Goal Activity:

  • Question Changes: How many changes are happening to the source code, during a certain time period?

    • Metric Code_Changes(Period): Number of changes to the source code (see Code_Changes).
    • Metric Code_Changes_Lines(Period): Aggregated number of lines touched in all changes (see Code_Changes_Lines).

  • Question Reviews: How many reviews to proposed changes to the source code are happening during a certain time period?

    • Metric Reviews(Period): Number of new review requests for changes to the source code (see Reviews).

    • Metric Reviews_Accepted(Period): Number of reviews for changes to the source code that ended accepting the change (see Reviews_Accepted).

    • Metric Reviews_Declined(Period): Number of reviews for changes to the source code that ended declining the change (see Reviews_Declined).

  • Question Issues: How many issues related to the source code are there during a certain time period?

    • Metric Issues(Period): Number of new issues related to the source code (see Issues_New).
    • Metric Issues_Active(Period): Number of issues related to the source code that showed some activity during the period (see Issues_Active).
    • Metric Issues_Closed(Period): Number of issues related to the source code that were closed (see Issues_Closed).

Goal Efficiency:

  • Question Reviews: How efficient is the project in reviewing proposed changes to the code, during a certain time period?

    • Metric Reviews_Duration(Period): For how long proposed changes are reviewed before they are accepted (see Reviews_Duration).
    • Metric Review_Acceptance(Period): How many reviews end accepting the code change.
    • Metric Review_Participants(Period): How many persons participated in reviews of code changes.
    • Metric: Review_Backlog(Period): How many reviews are still undecided (proposed changes were neither accepted nor declined)?
    • Summary metric: Review_Acceptance_Ratio(Period): Which fraction of new proposals for changes are finally accepted after a review process?
    • Summary metric: Review_Throughput(Period): How many proposals for changes are decided after code review (either accepted or declined) with respect to the number of proposals submitted?

  • Question Issues: How efficient is the project in dealing with issues related to the source code, for issues proposed during a certain time period?

    • Metric Issue_Duration(Period): Time since an issue is proposed until it is closed.
    • Metric Issue_Participants(Period): How many persons participated in the discussion of issues.
    • Metric: Issue_Backlog(Period): How many issues are still open?
    • Summary metric: Issue_Throughput(Period): How many issues are closed with respect to the number of issues opened?

Goal Quality:

  • Question Code_Review: Which fraction of the code goes through code review?

  • Question Testing: Which fraction of the code is tested?

Summary of metrics

This is the list of metrics that we consider as defined, even when some of them are still work in progress in some details:

Goal Question Metric
Activity Changes Code_Changes
Activity Changes Code_Changes_Lines
Activity Issues Issues_New
Activity Issues Issues_Active
Activity Issues Issues_Closed
Activity Reviews Reviews
Activity Reviews Reviews_Accepted
Activity Reviews Reviews_Declined
Efficiency Reviews Reviews_Duration

Legacy metrics and questions

We're in the process of discussing questions and metrics. Meanwhile, below is the former list of metrics, and their related questions.

Metric Question
Pull Requests Merged What is the number of pull requests merged?
Lines of Code Changed What is the number of lines of code changed?
Code Reviews What is the number of code reviews?
Pull Request Merge Duration What is the duration of time between code merge request and code commit?
Code Review Efficiency What is the number of merged code changes/number of abandoned code change requests?
Maintainer Response to Merge Request Duration What is the duration of time for a maintainer to make a first response to a code merge request?
Code Review Iteration What is the number of iterations that occur before a merge request is accepted or declined?
Forks Forks are a concept in distributed version control systems like GitHub. It is a proxy for the approximate number of developers who have taken a shot at building and deploying the codebase for development.
Pull Requests Open Number of open pull requests.
Pull Requests Closed Number of closed pull requests.
Pull Request Comment Duration The difference between the timestamp of the pull request creation date and the most recent comment on the pull request.
Pull Request Comment Diversity Number of each people discussing each pull request.
Pull Request Comments Number of comments on each pull request.

Disclaimer: The name/question pairs listed are not meant to represent a fully comprehensive list. It is expected that this list will evolve as people have insights and thoughts about the name/question pairs that comprise Growth-Maturity-Decline.

Tooling: The name/question pairs are intended to be a starting point for CHAOSS-related software. It is expected that this list will evolve based on the ability (or inability) of software to successfully implement the specific name/question pairs.

Background: The name/question pairs have been identified based on CHAOSS-related outreach activities. We thank everyone who participated.

How to contribute: