Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatically test all code examples in this repository. #255

Closed
Stebalien opened this issue Jul 29, 2019 · 6 comments
Closed

Automatically test all code examples in this repository. #255

Stebalien opened this issue Jul 29, 2019 · 6 comments
Labels
dif/expert Extensive knowledge (implications, ramifications) required effort/weeks Estimated to take multiple weeks kind/enhancement A net-new feature or an improvement to an existing feature need/analysis Needs further analysis before proceeding P3 Low: Not priority right now status/deferred Conscious decision to pause or backlog topic/docs Documentation topic/infra Infrastructure

Comments

@Stebalien
Copy link
Member

We have the following line in our release process checklist:

Ensure that all the examples we have produced for go-ipfs run without problems

This is simply impossible to achieve manually.


We should have doctests like in rust (https://doc.rust-lang.org/rustdoc/documentation-tests.html).

  1. All examples in documentation should be clearly marked.
  2. Each piece of documentation should include machine readable instructions for testing the documentation (e.g., the environment the document should be tested in). Personally, I'm thinking yaml top matter.
  3. We need something that actually runs these tests.

For example, every document could start with:

---
....
test: testing tool
---

We could then have a testing tool iterate over every markdown file, running the appropriate testing tool against each one.

@jessicaschilling
Copy link
Contributor

@Stebalien -- Thanks for creating this issue. For purposes of tracking docs-related issues across repos, do you mind adding the label "docs-ipfs" to new issues going forward? Thanks!

@jessicaschilling jessicaschilling changed the title Test Examples Content Enhancement: Test Examples Jul 31, 2019
@Stebalien
Copy link
Member Author

Got it. I've added an issue template to automatically apply that label to new issues filed against this repo.

@jessicaschilling
Copy link
Contributor

@Stebalien, thanks, you beat me to it! 😄

@jessicaschilling jessicaschilling changed the title Content Enhancement: Test Examples Implement automatic testing for documentation examples Sep 19, 2019
@johnnymatthews
Copy link
Contributor

This would be an incredible add to the whole IPFS org, but we'll have to put it on the back-burner for the moment. If there's serious push to get this sorted, we can tackle in in Q3.

@hsanjuan hsanjuan transferred this issue from ipfs-inactive/docs May 22, 2020
@johnnymatthews johnnymatthews changed the title Implement automatic testing for documentation examples Automatically test all code examples in this repository. Jun 18, 2020
@johnnymatthews johnnymatthews added dif/expert Extensive knowledge (implications, ramifications) required effort/weeks Estimated to take multiple weeks kind/enhancement A net-new feature or an improvement to an existing feature need/analysis Needs further analysis before proceeding P3 Low: Not priority right now status/deferred Conscious decision to pause or backlog topic/docs Documentation topic/infra Infrastructure labels Jun 18, 2020
@johnnymatthews johnnymatthews added this to the Automate the docs milestone Jun 18, 2020
@johnnymatthews
Copy link
Contributor

I've added the need/analysis tag to this issue. We need to have a look into how this process would work, and if we can add it into CircleCI (or if we'd need to run the tests locally before pushing GitHub).

@johnnymatthews johnnymatthews removed this from the Automate the docs milestone Oct 6, 2020
@johnnymatthews
Copy link
Contributor

This is overly ambitious. Instead, when we're showing the reader a code example, we copy it directly from https://github.com/ipfs-examples. Closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dif/expert Extensive knowledge (implications, ramifications) required effort/weeks Estimated to take multiple weeks kind/enhancement A net-new feature or an improvement to an existing feature need/analysis Needs further analysis before proceeding P3 Low: Not priority right now status/deferred Conscious decision to pause or backlog topic/docs Documentation topic/infra Infrastructure
Projects
None yet
Development

No branches or pull requests

3 participants