Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs Improvement: Introduce Change Log System #4946

Merged
merged 3 commits into from
Sep 6, 2023
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
130 changes: 98 additions & 32 deletions wiki/_README.md
Original file line number Diff line number Diff line change
@@ -1,65 +1,131 @@
_Certified fresh 230808 by Graham Johnson._
**Contents**
- [Navigating The Docs](#navigating-the-docs)
- [Updating The Docs: How & When](#updating-the-docs-how--when)
* [Documentation Language](#documentation-language)
+ [Indexicality & Timestamps](#indexicality--timestamps)
+ [Must, Should, & May](#must-should--may)
- [Tracking Documentation Status](#tracking-documentation-status)
* [Change Logs](#change-logs)
+ [Example Change Log](#example-change-log)
+ [Change Log Keyphrases](#change-log-keyphrases)
- [Authored By](#authored-by)
- [Certified Fresh By](#certified-by)
- [Flagged By](#flagged-by)
- [Merged By](#merged-by)
- [Split Off By](#split-off-by)
- [Updated By](#updated-by)
- [Change Log](#change-log)

# Navigating the docs

For a complete list of entries, see our [Table of Contents](_TOC.md)
For a complete list of wiki entries, see our [Table of Contents](_TOC.md)

# Updating the docs: How & when
# Updating The Docs: How & When

Whenever a code change PR breaks (or makes obsolete) existing documentation, the relevant entry should be updated. This documentation update should be included within the PR itself, similar to how one would update unit tests alongside a breaking code change.
Whenever engineering work makes existing documentation obsolete, relevant entries should be updated. This documentation update should be included within the PR itself, similar to how one would update unit tests alongside a breaking code change.

Similarly, whenever a PR adds codebase functionality where either:
Similarly, whenever a PR introduces new codebase functionality where either...

1. accompanying documentation has been explicitly requested
2. the implementing engineer believes the new functionality will not be easily legible to or extendable by future engineers
1. accompanying documentation has been explicitly requested
2. the implementing engineer believes the new functionality will not be easily comprehensible future engineers

...then that PR should include a new documentation entry as part of its purview.
...then that PR should include documentation as part of its purview.

At least for the duration of 2023, as we implement this new system, any PR that updates or contributes to our documentation wiki should tag @gdjohnson (the documentarian) on Github.
Significant updates to wiki entries should be manually listed in a given entry's change log; see the [Change Log](#change-log) sub-entry for guidelines and context. All entries should follow the [Documentation Language](#documentation-language) guidelines.

# Documentation language
At least for the duration of 2023, as we implement this new system, any PR that updates or contributes to our documentation wiki should tag @gdjohnson (the documentarian) on GitHub.

## Must, Should, & May
## Documentation Language

### Indexicality & Timestamps

Language is "indexical" insofar as it relies on inferred context to be understood. For instance, in our wiki, words like "I," "you," "now," and "soon" are indexical insofar as they rely on a context, known to the contributor at time of writing, that will likely be lost to future readers.

In place of pronouns, documentation should be written with proper names and nouns. Moreover, it is preferable to name _teams_ rather than individuals, since individuals may be replaced or change over time.

References to places and times (e.g. "now," "soon," "recently," "historically") should be explicitly contextualized with dates whenever possible (e.g. "As of 230831..."). Dates across our wiki and codebase should be structured YYMMDD, e.g. 230831 for August 31st, 2023.

### Must, Should, & May

Per [RFC 2119](https://www.rfc-editor.org/rfc/rfc2119), the following prescriptive language should be consistently used across documentation in order to specify the strength of given guideliens or requirements:

1. **MUST**: This word, or the term "REQUIRED," conveys that the definition is an absolute requirement of the specification.
2. **MUST NOT**: This phrase conveys that the definition is an absolute prohibition of the specification.
3. **SHOULD**: This word, or the adjective "RECOMMENDED", conveys that there may exist valid reasons in particular circumstances to ignore a particular item, but the full implications must be understood and carefully weighed before choosing a different course.
4. **SHOULD NOT** This phrase, or the phrase "NOT RECOMMENDED" conveys that there may exist valid reasons in particular circumstances when the particular behavior is acceptable or even useful, but the full implications should be understood and the case carefully weighed before implementing any behavior described with this label.
5. **MAY** This word, or the adjective "OPTIONAL", mean that an item is truly optional.
4. **SHOULD NOT**: This phrase, or the phrase "NOT RECOMMENDED" conveys that there may exist valid reasons in particular circumstances when the particular behavior is acceptable or even useful, but the full implications should be understood and the case carefully weighed before implementing any behavior described with this label.
5. **MAY**: This word, or the adjective "OPTIONAL", mean that an item is truly optional.

# Tracking documentation status
# Tracking Documentation Status

An important part of maintaining a working documentation system involves flagging (1) areas in need of documentation (2) areas whose documentation has gone stale.

For now, the best system is notifying the archivist, Graham Johnson, on Slack or Github (@gdjohnson). Down the line, a more structured system may be worth implementing.
As of 230901, the best system for flagging stale and missing documentation is by either (1) notifying the archivist, Graham Johnson, on Slack or Github (@gdjohnson), or (2) adding to our [documentation wishlist](./_Wishlist.md). Down the line, a more structured system may be needed.

## Change Logs

Our git history has been frequently compromised across the history of the codebase, and is not fully reliable. Moreover, git history omits documentation audits and conflates minor local changes with noteworthy, page-wide updates.

In order to track the history and status of documentation (e.g. to determine whether an entry is "fresh" or "stale"), we're introducing manual change logs. Change logs must be placed at the bottom of each entry, under an H1 header. Change logs should record the following actions:
- Page authorship (using the keyphrase "Authored by")
- Significant rewrites, either of the entire entry, or dedicated sections (using the keyphrase "Updated by")
- Merges and branchings (using the keyphrases "Merged by," "Split off by")
- Audits and certifications (using the keyphrase "Certified fresh by")
- Flagging sections as incomplete, unclear, misleading, or out of date (using the keyphrase "Flagged by")

Change log entries should center around one of the default change log keyphrases, [listed below](#change-log-keyphrases). They should include additional useful context, including any relevant PRs, and must name the individual responsible for the action as well as the date the action occurred.

The change log should often be the first section of a wiki entry checked. Is the entry out of date? Has it been flagged for an overhaul? One goal of change logs is to prevent engineers from wasting time implementing obsolete documentation. A second goal is to ensure engineers, when assigned work in novel areas of the codebase, can be guaranteed adequate documentation for onboarding into that area.

### Example Change Log

```md
# Change Log

- 230606: Updated and certified fresh by Jane Doe, in light of API changes (#3150).
- 230403: Flagged by John Smith; will likely fall out of date with upcoming API changes.
- 221201: Certified fresh by Jane Doe.
- 220507: Authored by John Smith (#2350).
```

### Change Log keyphrases

#### Authored By

The author responsible for creating the entry. Since new wiki pages should only be created ~fully formed (i.e. no stubs or placeholders), the author will in most cases also be the individual with most knowledge of the area affected, or who is taking on some level of responsibility for the page's content.

Ideally, all newly authored documentation should also be [certified fresh](#certified-fresh), and authorship on a given date should be assumed to be synonymous with certification on that date. However, as of 230901, we should specifically include the "certified fresh" keyphrase alongside the "authored by" entry, to guarantee this standard of quality.

#### Certified Fresh By

When a contributor certifies an entry as fresh, he is attesting that he has successfully tested, or else followed (implemented, executed), the entry on a given date. For non-codebase process documentation (e.g. [Ways of Working](./Ways-Of-Working.md)), an entry is certified fresh when an authorized individual (typically a lead) has reviewed and signed off on an entry's guidelines.

All newly added pages or sections in the wiki should be explicitly certified to the date of their addition, as it is assumed that all new documentation meets the above criteria.

A contributor may certify either a full entry, or a section of that entry, as fresh. But it is important not to certify a wider scope than has been actually verified as accurate. Certification is meant to be a strong signal to engineers, in using documentation, that the documentation can be relied upon as up-to-date and accurate. Dilution of the certification system is more harmful than a lack of certification.

The other purpose of certification is to provide an additional contact, so that users of the documentation can reach out with questions or comments. (Authors are typically assumed to be the primary contact for entries.)

## "Certified fresh" datestamps
#### Flagged By

Moving forward, our goal is to keep a “Certified fresh” datestamp on all wiki entries. This timestamp is accompanied by an optional note and the name of the certifier. For consistency, dates should take the format YYMMDD.
If an engineer is using or auditing the docs, and notices an incomplete, out-of-date, or misleading entry, this should be explicitly flagged.

Example certification:
> _Certified fresh 230731 by Graham Johnson._
> _Note: The optional super-admin steps listed under the "Verifying addresses" sub-section were not re-tested and may be stale._
Any concerns about future reliability of the entry (e.g. an engineer knows that in a few months, a new technical solution will replace the currently documented solution) should be noted here whenever possible.

### What does “Certified fresh” mean operationally?
The goal of the "Flagged by" keyphrase is to raise awareness of areas of the wiki need attention, and to prevent future engineers from implementing misleading or out-of-date instructions.

1. "Certified fresh on X date" means that an engineer or documentarian has successfully tested or followed the entry’s documentation on X date.
2. Successfully acting upon a few tidbits of information on an entry page for development guidance is not sufficient to certify an entry.
3. "Certified fresh" is a _strong recommendation_ from the certifying team member that the documentation can be faithfully followed.
4. All newly added pages or sections in the wiki should be explicitly certified to the date of their addition, as it is assumed that all new documentation should meet the above criteria (#s 1-3).
#### Merged By

### Certification notes & elaboration
If pages are merged together, this should be noted in the change log of the merged-into page, and a compressive summary of the merged page should be provided below.

Any additional information (e.g. caveats, conditionals, environmental details that might make the certification partial or provisional) should be added as a note under the datestamp.
#### Split Off By

If only a sub-section can be certified fresh, that section should be explicitly noted. The same is true of any sub-sections that should be excluded from the certification (e.g. that the certifier cannot vouch for, has been unable to test, etc).
If a section is moved out of one entry and into another, this should be listed explicitly in the change logs, so that engineers who have come to rely on the section's previous location are able to find it.

Any concerns about future reliability of the entry (e.g. the certifying engineer knows that in a few months, a new technical solution will replace the currently documented solution) should be noted here if possible.
#### Updated By

## Certifying: When & how
Used for all significant updates or changes of note. This helps flag, for engineers who may have recently used a docs entry, that some load-bearing aspect of documentation has changed.

All certifications (new or updated) should be either (1) submitted in the larger PR in which the documentation entry was certified fresh (2) be opened as a separate PR.
# Change Log

At least for the duration of 2023, as we implement this new system, all certifications and additions to documentation should tag/flag @gdjohnson (the documentarian) on Github
- 230831: Updated and certified fresh by Graham Johnson (#4941).
- 230808: Authored by Graham Johnson.