Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: add pre-commit config and run linters #62

Merged
merged 6 commits into from
Feb 23, 2024

Conversation

lmmx
Copy link
Contributor

@lmmx lmmx commented Jan 11, 2024

Background: Follow-up to #58, which was cancelled as it combined the contribution of pre-commit config (to run linters) with a refactor (for code structure), and it transpired that only the former was desirable upon submission

I've re-done the changes (by copying over the pre-commit exactly as in the previous PR #58 and executing it on this fresh branch). The only edits that were required to achieve a successful linting were the removal of * imports (torchvision.transforms.* and uform.models.* specifically)

Other than that nothing was changed, then the linters ran.

Some of these are the built-in ones from the pre-commit tool's own hooks:

  • check-toml
  • check-yaml
  • debug-statements (to avoid committing breakpoints etc)
  • end-of-file-fixer (standardise newlines at end of file)
  • name-tests-test (not needed yet as there are no pytest tests but you may wish to include them in future so no harm?)
  • trailing-whitespace

Additionally:

  • toml-sort-fix (lints pyproject.toml)
  • ruff (replaces many other tools which are selected with single letter rule category codes in pyproject.toml: pyflakes, pycodestyle, isort, pyupgrade)

To run this on CI all you have to do is go to pre-commit.ci and give its GitHub Actions app permission to run on your CI (does not require a .github/workflows YAML config to run). The pre-commit config YAML file introduced in this PR is sufficient to configure the CI workflow to then run it.

@lmmx lmmx marked this pull request as ready for review January 11, 2024 13:03
@ashvardanian ashvardanian changed the base branch from main to main-dev January 11, 2024 20:09
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be a good idea to move this to the .github folder to the rest of GitHub-specific configs?

Copy link
Contributor Author

@lmmx lmmx Jan 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The .pre-commit-config.yaml is not GitHub specific, it's git specific, and should live at the project root

Adding pre-commit plugins to your project ¶

Add a file called .pre-commit-config.yaml to the root of your project. The pre-commit config file describes what repositories and hooks are installed.

Once you've put it there it can be run from anywhere with pre-commit run --all-files or [the typical use] you can run pre-commit install which "installs" the hooks into the .git/hooks directory of your cloned git repo directory so that they run automatically whenever you git commit [this step is not strictly necessary if you let the CI bot run them and make "autofix commits", which we do here].

The autofix functionality is documented on the pre-commit.ci website:

auto fixing pull requests: if tools make changes to files during a pull request, pre-commit.ci will automatically fix the pull request.

limitations under the License.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to exclude this file from linting? I'm afraid some of the GitHub infra may consider this a custom license even if we add as much as a newline.

Copy link
Contributor Author

@lmmx lmmx Jan 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. It is possible to exclude from linting using an exclude pattern (see docs)
  2. You needn't be afraid of that, this edit was introduced by the "end of file fixer" which I also use in other projects (e.g. fugit which do not invalidate upon linting.

Licenses aren't detected in a fragile way (like a MD5 checksum/exact match), we can tell that from e.g. MIT licensed projects where the license is detected despite the user editing it (you have to put your name in it for it to be valid).

Specifically the docs on how licenses are detected on GitHub using the open source Ruby gem licensee using several "matchers" which notes that its method includes

If we strip away whitespace and copyright notice,

Which confirms that the whitespace change is not going to obscure the license [it does in fact do more than that using string matching but as I say that wouldn't become relevant]

@ashvardanian ashvardanian merged commit 0a3efac into unum-cloud:main-dev Feb 23, 2024
ashvardanian pushed a commit that referenced this pull request Feb 23, 2024
## [1.1.1](v1.1.0...v1.1.1) (2024-02-23)

### Docs

* Performance observations for M2 CPUs (#56) ([8374ef6](8374ef6)), closes [#56](#56)

### Fix

* Passing labels to `text_decoder` to compute loss. (#65) ([f445a8b](f445a8b)), closes [#65](#65)

### Improve

* Larger batch benchmarks ([fdc8587](fdc8587))

### Make

* pre-commit config and linters (#62) ([0a3efac](0a3efac)), closes [#62](#62)
@ashvardanian
Copy link
Contributor

🎉 This PR is included in version 1.1.1 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants