Implement rules in `darglint` #458

edgarrmondragon · 2022-10-21T01:21:21Z

This might be superseded by #12434

Darglint's primary focus is to identify incorrect and missing documentationd
of a function's signature. Checking style is a stretch goal, and is supported
on a best-effort basis. Darglint does not check stylistic preferences expressed
by tools in the Python Code Quality Authority (through tools such as pydocstyle).
So when using Darglint, it may be a good idea to also use pydocstyle, if you
want to enforce style. (For example, pydocstyle requires the short summary
to be separated from other sections by a line break. Darglint makes no such check.)

Error Codes

DAR001: The docstring was not parsed correctly due to a syntax error.
DAR002: An argument/exception lacks a description
DAR003: A line is under-indented or over-indented.
DAR004: The docstring contains an extra newline where it shouldn't.
DAR005: The item contains a type section (parentheses), but no type.
DAR101: The docstring is missing a parameter in the definition.
DAR102: The docstring contains a parameter not in function.
DAR103: The docstring parameter type doesn't match function.
DAR104: (disabled) The docstring parameter has no type specified
DAR105: The docstring parameter type is malformed.
DAR201: The docstring is missing a return from definition.
DAR202: The docstring has a return not in definition.
DAR203: The docstring parameter type doesn't match function.
DAR301: The docstring is missing a yield present in definition.
DAR302: The docstring has a yield not in definition.
DAR401: The docstring is missing an exception raised.
DAR402: The docstring describes an exception not explicitly raised.
DAR501: The docstring describes a variable which is not defined.

The text was updated successfully, but these errors were encountered:

charliermarsh · 2022-10-29T22:55:56Z

Cool, I've never used but looks useful. Would this be an impactful plugin for you? Would it unblock any Ruff adoption? :)

edgarrmondragon · 2022-11-01T00:00:09Z

Hey @charliermarsh, yeah. I maintain a few packages with public APIs that would benefit from contributors being informed about documentation for arguments or exceptions raised being missing.

A number of features would make Ruff a nice replacement for flake8: speed, compatibility with the rest of the Python ecosystem, pyproject.toml support.

lsorber · 2022-11-18T14:32:07Z

FWIW, darglint is the main blocker [1] for us to migrate Poetry Cookiecutter [2] from flake8 to ruff.

[1] superlinear-ai/poetry-cookiecutter#125
[2] https://github.com/radix-ai/poetry-cookiecutter

charliermarsh · 2022-11-18T15:07:56Z

👍 Helpful to hear! darglint is probably the most popular unimplemented plugin. I'd like to get around to it soon. If there's a subset of rules that are most impactful, that'd be helpful to know too for prioritization.

edgarrmondragon · 2022-11-19T00:03:32Z

If there's a subset of rules that are most impactful, that'd be helpful to know too for prioritization.

For me personally, they're probably DAR101, DAR102, DAR201, DAR202 and to a lesser extent DAR301, DAR302, DAR401 and DAR402

strickvl · 2023-01-04T09:52:43Z

For me, I use darglint together with pydocstyle since they don't do quite the same thing. Having darglint absorbed into the ruff world would be really great.

erwann-met · 2023-01-10T10:08:44Z

We just discovered ruff in my team. It's amazing, and I'd like to thank you for all the great work!

We used to run flake8 along with darglint. Unfortunately since ruff does not yet implement darglint rules we still have to keep darglint alongside ruff which is a shame given how slow darglint is.

Reimplementing darglint within ruff is definitely the feature we are most looking forward to!

For us the most important rules are probably those related to the parameters (and returns) and their types: DAR101, DAR102, DAR103, DAR201, DAR202, DAR203 and DAR501. I hope this helps!

We understand that maintaining an open-source project can be a lot of work and we want to express our appreciation for all the effort that goes into it. We were wondering if there's any update on when this issue might be addressed? Any information would be greatly appreciated. Thank you!

charliermarsh · 2023-01-10T13:05:39Z

Very helpful, thank you! It's not something I've started working on actively (and would make for a good contribution if there are any eager contributors reading this :)), but that set of rules at least is probably not too much work to support, and this issue has a lot of thumbs-up votes so I'll try to get to it soon-ish. I'll post here when I'm able to take it on.

charliermarsh · 2023-01-11T04:05:37Z

Looking into starting on this. darglint has a lot of code because it defines entire grammars for the docstrings, and then lexes and parses according to those grammars. pydocstyle, on the other hand, does it all with regular expressions. Wondering if we could get away with the latter.

anthonyburdi · 2023-01-25T19:26:54Z

Super exciting to hear that this is in the works. We just switched to ruff instead of pydocstyle and are eager to replace darglint as well. Here are the error codes we are most interested (least overlap with pydocstyle):

DAR102: The docstring contains a parameter not in function.
DAR201: The docstring is missing a return from definition.
DAR202: The docstring has a return not in definition.
DAR301: The docstring is missing a yield present in definition.
DAR302: The docstring has a yield not in definition.
DAR401: The docstring is missing an exception raised.
DAR402: The docstring describes an exception not explicitly raised.
DAR501: The docstring describes a variable which is not defined.

rbebb · 2023-02-02T00:30:19Z

It would be great if it could add darglint’s feature that checks if docstrings match a particular style (whether it be Sphinx, Google, or something else)

baggiponte · 2023-03-07T17:41:52Z

Implementing darglint could handle some flake8 exceptions, as well as part of docformatter I guess?

rbebb · 2023-03-27T13:58:30Z

@charliermarsh Is there an update on implementing darglint features in ruff? Thank you for all of your work on this project!

finswimmer · 2023-04-06T06:50:58Z

Hey,

unfortunately darglint is unmaintained since December 2022. So I would be very happy to see something similar in ruff.

Thanks a lot for all your very good work!

fin swimmer

charliermarsh · 2023-04-06T17:46:54Z

I'm currently considering taking this on as a project over the next few weeks, but it's competing with some other priorities so not a firm commitment yet :) I definitely hear that this something users want, and it's something I want Ruff to support too -- just a question of when, will post here when I have a clear decision.

leandro-lucarella-frequenz · 2023-04-12T18:24:04Z

Great to hear work is close to being started. We also rely a lot on darglint and are concerned about is current abandoned state, and were also looking to start using ruff, so it would be very good news if it can be a replacement for darglint. The rules that most often catch error in the documentation for us are:

DAR101: The docstring is missing a parameter in the definition.
DAR102: The docstring contains a parameter not in function.
DAR201: The docstring is missing a return from definition.
DAR202: The docstring has a return not in definition.
DAR301: The docstring is missing a yield present in definition.
DAR302: The docstring has a yield not in definition.
DAR401: The docstring is missing an exception raised.
DAR402: The docstring describes an exception not explicitly raised.

ddelange · 2023-10-25T05:29:51Z

Switched from flake8 + black + darglint to ruff + pydoclint.

Note that for google style docstrings:

pydoclint needs additional config beyond style=google (below, see also discussion) if your code is already type annotated (which doesnt need duplication into the docstrings)
D407 needs to be disabled.

pyproject.toml

[tool.pydoclint]
style = "google"
arg-type-hints-in-docstring = false
check-return-types = false
check-yield-types = false

[tool.ruff]
select = ["ALL"]
ignore = ["D407", "E501", "ANN", "TRY003", "D203", "D213", "D100", "D104", "D106", "FIX002", "ERA001", "RUF012"] # ignores: D407 (we have google style docstrings), E501 (we have black), ANN (we have mypy), TRY003 (there is EM102), D203 (there is D211), D213 (there is D212), D100,D104 (we don't publish publick readthedocs), D106 (we have Django Meta class that doesn't need a docstring), FIX002 (we have TD002,TD003), ERA001 (we sometimes leave code for reference), RUF012 (django is full of class vars that are lists or dicts)
target-version = "py311" # not needed if your pyproject.toml has `project.requires-python`

[tool.ruff.extend-per-file-ignores]
"__init__.py" = ["E401", "E402"]
"**/tests/**/*.py" = ["S101", "PT009"] # ignores: S101 (assert is fine in tests), PT009 (we use django unittest framework)
"**/migrations/*.py" = ["D101"]

.pre-commit-config.yaml

repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
  rev: v0.1.0
  hooks:
  - id: ruff
    args: [--fix, --exit-non-zero-on-fix]
  - id: ruff-format

# should be replaced in the future ref https://github.com/astral-sh/ruff/issues/458
- repo: https://github.com/jsh9/pydoclint
  rev: 0.3.4
  hooks:
  - id: pydoclint

# should be replaced in the future ref https://github.com/astral-sh/ruff/issues/3792
- repo: https://github.com/asottile/blacken-docs
  rev: 1.13.0
  hooks:
  - id: blacken-docs
    additional_dependencies: ['black==23.10.0']

BigForLy · 2023-12-20T07:08:55Z

Hello @charliermarsh! I manage to convert almost all checks to ruff, except for Darling, do you have any idea whether this will be done?

strickvl · 2024-02-27T12:55:03Z

As mentioned in Discord, registering my interest in taking a stab at this.

Spenhouet · 2024-04-16T11:44:43Z

Using pydocstyle we are using the google style with typing. We strictly use typing in python, so repeating the types in the Doc string serves no purpose and is just redundant. Having rules like DAR103: The docstring parameter type doesn't match function. or DAR105: The docstring parameter type is malformed. certainly would help to keep Doc string and parameter declaration aligned, it's still redundant and we'd rather enforce that no types are specified in the Doc string. Seems darglint doesn't provide such a rule.

Edit: Just found this issue which raises the same wish: #9070

AloizioMacedo · 2024-04-16T14:41:26Z

Hoping to see this as well! Ruff already encompasses most of my CI linting checks, this would be a great addition.

Meanwhile, in case it is useful for other people, I decided to take a stab and implement a version that is sufficient for my needs: https://github.com/AloizioMacedo/pystaleds.

It intends primarily to check for mismatches of the parameters in the function signature and the arguments in the docstring. It is very experimental, but I can already use it in my projects while ruff doesn't support a similar feature : )

ddelange · 2024-04-16T15:15:30Z

we'd rather enforce that no types are specified in the Doc string

@Spenhouet fwiw pydoclint has an --arg-type-hints-in-signature flag ref https://jsh9.github.io/pydoclint/violation_codes.html

see also my config above

Spenhouet · 2024-04-17T06:16:41Z

@ddelange Thanks for sharing, that would introduce a new tool just for this small requirement. Would much more prefer if this would be covered by a linting rule within ruff.

aeturrell · 2024-05-15T07:21:18Z

Just wanted to add my voice to those saying that this would be a really useful feature. Thanks for all the amazing work on Ruff!

tmke8 · 2024-05-21T14:30:55Z

I wonder if at this point, it would be better to implement pydoclint in ruff instead of darglint? (This is prompted by the recent PR #11471.)

Pydoclint has

more error codes (e.g., DOC104 in pydoclint checks for the order of the parameters; darglint doesn't have that AFAICT),
more config options (like arg-type-hints-in-signature which lets you decide whether you want to include types in the docstring),
and the config options of pydoclint also seem more well-thought-out to me than darglint's. (E.g., pydoclint has skip-checking-short-docstrings which makes sense to me, but darglint's strictness=short vs strictness=long is a bit confusing)

Goldziher · 2024-07-15T15:40:34Z

I wonder if at this point, it would be better to implement pydoclint in ruff instead of darglint? (This is prompted by the recent PR #11471.)

Pydoclint has

more error codes (e.g., DOC104 in pydoclint checks for the order of the parameters; darglint doesn't have that AFAICT),

more config options (like arg-type-hints-in-signature which lets you decide whether you want to include types in the docstring),

and the config options of pydoclint also seem more well-thought-out to me than darglint's. (E.g., pydoclint has skip-checking-short-docstrings which makes sense to me, but darglint's strictness=short vs strictness=long is a bit confusing)

Strong +1 for this. pydoclint is really great and pretty fast already, but ruff could improve on this and extend it further. Also some issues might be autofixable.

charliermarsh · 2024-07-17T16:50:48Z

Are there rules that are present in darglint but not pydoclint?

Apakottur · 2024-07-17T16:54:37Z

Are there rules that are present in darglint but not pydoclint?

I know of at least two:

DAR401: The docstring is missing an exception raised.
DAR402: The docstring describes an exception not explicitly raised.

These two rules are the only reason we use darglint, everything else that we want is covered by pydoclint.

charliermarsh · 2024-07-17T16:55:23Z

Interesting, ok. Those are the exact rules added in #11471.

jsh9 · 2024-07-17T17:03:22Z

Are there rules that are present in darglint but not pydoclint?

I know of at least two:

DAR401: The docstring is missing an exception raised.

DAR402: The docstring describes an exception not explicitly raised.

These two rules are the only reason we use darglint, everything else that we want is covered by pydoclint.

I'm the author of pydoclint.

I think these two pydoclint violation codes cover DAR401 and 402:

Code	Explanation
DOC501	Function/method has “raise” statements, but the docstring does not have a “Raises” section
DOC502	Function/method has a “Raises” section in the docstring, but there are not “raise” statements in the body

If they don't cover what you want, please feel free to open an issue.

AndydeCleyre · 2024-07-18T15:53:23Z

@jsh9 I don't think those check the exception types. For example, if I have a function like this, which sometimes raises a ValueError, but incorrectly document that it raises a ZeroDivisionError, darglint will report the problem while pydoclint won't:

def _str_to_bool(informal_bool: str) -> bool:
    """
    Translate a commonly used boolean ``str`` into a real ``bool``.

    Args:
        informal_bool: A boolean represented as ``str``, like ``"true"``, ``"no"``, ``"off"``, etc.

    Returns:
        ``True`` or ``False`` to match the intent of ``informal_bool``.

    Raises:
        ZeroDivisionError: This doesn't look like enough like a ``bool`` to translate.
    """
    if informal_bool.lower() in ('true', 't', 'yes', 'y', 'on', '1'):
        return True
    if informal_bool.lower() in ('false', 'f', 'no', 'n', 'off', '0'):
        return False
    raise ValueError(f"{informal_bool} doesn't look like a boolean")  # pragma: no cover

$ darglint nt2
nt2/casters.py:_str_to_bool:29: DAR401: -r ValueError
nt2/casters.py:_str_to_bool:39: DAR402: +r ZeroDivisionError

$ pydoclint nt2
Loading config from user-specified .toml file: pyproject.toml
No config found in pyproject.toml.
Skipping files that match this pattern: \.git|\.tox
nt2/__init__.py
nt2/casters.py
nt2/converters.py
nt2/dumpers.py
nt2/ui.py
nt2/yamlpath_tools.py
🎉 No violations 🎉

jsh9 · 2024-07-18T20:48:45Z

Thanks @AndydeCleyre !

I've opened a new issue (jsh9/pydoclint#158) on pydoclint to add this feature.

…extraneous-exception` (`DOC501`, `DOC502`) (#11471) ## Summary These are the first rules implemented as part of #458, but I plan to implement more. Specifically, this implements `docstring-missing-exception` which checks for raised exceptions not documented in the docstring, and `docstring-extraneous-exception` which checks for exceptions in the docstring not present in the body. ## Test Plan Test fixtures added for both google and numpy style.

charliermarsh added the rule Implementing or modifying a lint rule label Oct 21, 2022

charliermarsh mentioned this issue Nov 26, 2022

Add flake8-debugger #909

Merged

lsorber mentioned this issue Dec 19, 2022

feat: replace flake8 with ruff superlinear-ai/poetry-cookiecutter#144

Merged

4 tasks

charliermarsh added docstring Related to docstring linting or formatting plugin Implementing a known but unsupported plugin and removed rule Implementing or modifying a lint rule labels Dec 31, 2022

lsorber mentioned this issue Jan 7, 2023

Add darglint once integrated in ruff superlinear-ai/poetry-cookiecutter#151

Open

thatlittleboy mentioned this issue Jan 30, 2023

[INF] darglint is archived -> move to ruff? pyjanitor-devs/pyjanitor#1236

Closed

charliermarsh mentioned this issue Jan 30, 2023

D417 Argument descriptions only required if there is an "Args" section #2310

Open

edgarrmondragon mentioned this issue Feb 2, 2023

Add Darglint Checks #2459

Closed

bashirmindee mentioned this issue Feb 27, 2023

[FEATURE] replace isort,black,flake8 with ruff mindee/tawazi#85

Closed

edgarrmondragon mentioned this issue Feb 27, 2023

ci: Use Ruff to lint project meltano/sdk#1462

Merged

leandro-lucarella-frequenz mentioned this issue Apr 12, 2023

Replace darglint with ruff or pydoclint frequenz-floss/frequenz-repo-config-python#18

Closed

ssbarnea mentioned this issue Apr 24, 2023

Enable darglint via pre-commit ansible-community/antsichaut#22

Merged

ireneisdoomed mentioned this issue Oct 31, 2023

feat: use pydoclint as a documentation linter opentargets/gentropy#212

Merged

sbrugman mentioned this issue Dec 10, 2023

Autofix remove types from docstring #9070

Open

strickvl mentioned this issue Jan 17, 2024

Add missing docstring for Skypilot setting zenml-io/zenml#2305

Merged

9 tasks

MarcoGorelli mentioned this issue Feb 29, 2024

CI: Docstring validation is slow pandas-dev/pandas#57578

Open

This was referenced May 18, 2024

pydocstyle: undocumented returns #11435

Closed

[pydoclint] Implement docstring-missing-exception and docstring-extraneous-exception (DOC501, DOC502) #11471

Merged

This was referenced May 23, 2024

Move pydoclint to flake8 ansible/ansible-dev-environment#161

Closed

Enable pydoclint via flake 8 ansible/ansible-dev-environment#165

Merged

bruno-fs mentioned this issue May 28, 2024

Proposal: use ruff to replace flake8 and related extensions cjolowicz/cookiecutter-hypermodern-python#1282

Open

cidrblock mentioned this issue Jun 26, 2024

Restore pydoclint, add class docstrings ansible/molecule#4236

Closed

jsh9 mentioned this issue Jul 18, 2024

Add a feature to check exceptions (name and order) jsh9/pydoclint#158

Closed

augustelalande mentioned this issue Jul 21, 2024

Implement pydoclint #12434

Open

35 tasks

strickvl mentioned this issue Sep 13, 2024

darglint becomes pydoclint zenml-io/zenml#3012

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement rules in `darglint` #458

Implement rules in `darglint` #458

edgarrmondragon commented Oct 21, 2022 •

edited

Loading

charliermarsh commented Oct 29, 2022

edgarrmondragon commented Nov 1, 2022

lsorber commented Nov 18, 2022

charliermarsh commented Nov 18, 2022

edgarrmondragon commented Nov 19, 2022

strickvl commented Jan 4, 2023

erwann-met commented Jan 10, 2023 •

edited

Loading

charliermarsh commented Jan 10, 2023

charliermarsh commented Jan 11, 2023

anthonyburdi commented Jan 25, 2023 •

edited

Loading

rbebb commented Feb 2, 2023

baggiponte commented Mar 7, 2023

rbebb commented Mar 27, 2023 •

edited

Loading

finswimmer commented Apr 6, 2023

charliermarsh commented Apr 6, 2023

leandro-lucarella-frequenz commented Apr 12, 2023

ddelange commented Oct 25, 2023 •

edited

Loading

BigForLy commented Dec 20, 2023

strickvl commented Feb 27, 2024

Spenhouet commented Apr 16, 2024 •

edited

Loading

AloizioMacedo commented Apr 16, 2024

ddelange commented Apr 16, 2024

Spenhouet commented Apr 17, 2024

aeturrell commented May 15, 2024

tmke8 commented May 21, 2024 •

edited

Loading

Goldziher commented Jul 15, 2024

charliermarsh commented Jul 17, 2024

Apakottur commented Jul 17, 2024

charliermarsh commented Jul 17, 2024

jsh9 commented Jul 17, 2024

AndydeCleyre commented Jul 18, 2024

jsh9 commented Jul 18, 2024

Implement rules in darglint #458

Implement rules in darglint #458

Comments

edgarrmondragon commented Oct 21, 2022 • edited Loading

Error Codes

charliermarsh commented Oct 29, 2022

edgarrmondragon commented Nov 1, 2022

lsorber commented Nov 18, 2022

charliermarsh commented Nov 18, 2022

edgarrmondragon commented Nov 19, 2022

strickvl commented Jan 4, 2023

erwann-met commented Jan 10, 2023 • edited Loading

charliermarsh commented Jan 10, 2023

charliermarsh commented Jan 11, 2023

anthonyburdi commented Jan 25, 2023 • edited Loading

rbebb commented Feb 2, 2023

baggiponte commented Mar 7, 2023

rbebb commented Mar 27, 2023 • edited Loading

finswimmer commented Apr 6, 2023

charliermarsh commented Apr 6, 2023

leandro-lucarella-frequenz commented Apr 12, 2023

ddelange commented Oct 25, 2023 • edited Loading

BigForLy commented Dec 20, 2023

strickvl commented Feb 27, 2024

Spenhouet commented Apr 16, 2024 • edited Loading

AloizioMacedo commented Apr 16, 2024

ddelange commented Apr 16, 2024

Spenhouet commented Apr 17, 2024

aeturrell commented May 15, 2024

tmke8 commented May 21, 2024 • edited Loading

Goldziher commented Jul 15, 2024

charliermarsh commented Jul 17, 2024

Apakottur commented Jul 17, 2024

charliermarsh commented Jul 17, 2024

jsh9 commented Jul 17, 2024

AndydeCleyre commented Jul 18, 2024

jsh9 commented Jul 18, 2024

Implement rules in `darglint` #458

Implement rules in `darglint` #458

edgarrmondragon commented Oct 21, 2022 •

edited

Loading

erwann-met commented Jan 10, 2023 •

edited

Loading

anthonyburdi commented Jan 25, 2023 •

edited

Loading

rbebb commented Mar 27, 2023 •

edited

Loading

ddelange commented Oct 25, 2023 •

edited

Loading

Spenhouet commented Apr 16, 2024 •

edited

Loading

tmke8 commented May 21, 2024 •

edited

Loading