Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pyupgrade] Restore the keep-runtime-typing setting #5470

Merged
merged 1 commit into from
Jul 3, 2023
Merged

[pyupgrade] Restore the keep-runtime-typing setting #5470

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions BREAKING_CHANGES.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,31 @@
# Breaking Changes

## 0.0.276

### The `keep-runtime-typing` setting has been reinstated ([#5470](https://github.com/astral-sh/ruff/pull/5470))

The `keep-runtime-typing` setting has been reinstated with revised semantics. This setting was
removed in [#4427](https://github.com/astral-sh/ruff/pull/4427), as it was equivalent to ignoring
the `UP006` and `UP007` rules via Ruff's standard `ignore` mechanism.

Taking `UP006` (rewrite `List[int]` to `list[int]`) as an example, the setting now behaves as
follows:

- On Python 3.7 and Python 3.8, setting `keep-runtime-typing = true` will cause Ruff to ignore
`UP006` violations, even if `from __future__ import annotations` is present in the file.
While such annotations are valid in Python 3.7 and Python 3.8 when combined with
`from __future__ import annotations`, they aren't supported by libraries like Pydantic and
FastAPI, which rely on runtime type checking.
- On Python 3.9 and above, the setting has no effect, as `list[int]` is a valid type annotation,
and libraries like Pydantic and FastAPI support it without issue.

In short: `keep-runtime-typing` can be used to ensure that Ruff doesn't introduce type annotations
that are not supported at runtime by the current Python version, which are unsupported by libraries
like Pydantic and FastAPI.

Note that this is not a breaking change, but is included here to complement the previous removal
of `keep-runtime-typing`.

## 0.0.268

### The `keep-runtime-typing` setting has been removed ([#4427](https://github.com/astral-sh/ruff/pull/4427))
Expand Down
12 changes: 9 additions & 3 deletions crates/ruff/src/checkers/ast/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2108,6 +2108,7 @@ where
&& self.settings.target_version >= PythonVersion::Py37
&& !self.semantic.future_annotations()
&& self.semantic.in_annotation()
&& !self.settings.pyupgrade.keep_runtime_typing
{
flake8_future_annotations::rules::future_rewritable_type_annotation(
self, value,
Expand All @@ -2118,7 +2119,8 @@ where
if self.settings.target_version >= PythonVersion::Py310
|| (self.settings.target_version >= PythonVersion::Py37
&& self.semantic.future_annotations()
&& self.semantic.in_annotation())
&& self.semantic.in_annotation()
&& !self.settings.pyupgrade.keep_runtime_typing)
{
pyupgrade::rules::use_pep604_annotation(
self, expr, slice, operator,
Expand Down Expand Up @@ -2216,6 +2218,7 @@ where
&& self.settings.target_version >= PythonVersion::Py37
&& !self.semantic.future_annotations()
&& self.semantic.in_annotation()
&& !self.settings.pyupgrade.keep_runtime_typing
{
flake8_future_annotations::rules::future_rewritable_type_annotation(
self, expr,
Expand All @@ -2226,7 +2229,8 @@ where
if self.settings.target_version >= PythonVersion::Py39
|| (self.settings.target_version >= PythonVersion::Py37
&& self.semantic.future_annotations()
&& self.semantic.in_annotation())
&& self.semantic.in_annotation()
&& !self.settings.pyupgrade.keep_runtime_typing)
{
pyupgrade::rules::use_pep585_annotation(
self,
Expand Down Expand Up @@ -2291,6 +2295,7 @@ where
&& self.settings.target_version >= PythonVersion::Py37
&& !self.semantic.future_annotations()
&& self.semantic.in_annotation()
&& !self.settings.pyupgrade.keep_runtime_typing
{
flake8_future_annotations::rules::future_rewritable_type_annotation(
self, expr,
Expand All @@ -2301,7 +2306,8 @@ where
if self.settings.target_version >= PythonVersion::Py39
|| (self.settings.target_version >= PythonVersion::Py37
&& self.semantic.future_annotations()
&& self.semantic.in_annotation())
&& self.semantic.in_annotation()
&& !self.settings.pyupgrade.keep_runtime_typing)
{
pyupgrade::rules::use_pep585_annotation(self, expr, &replacement);
}
Expand Down
103 changes: 103 additions & 0 deletions ...rules/numpy/snapshots/ruff__rules__numpy__tests__numpy-deprecated-function_NPY003.py.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ NPY003.py:5:1: NPY003 [*] `np.cumproduct` is deprecated; use `np.cumprod` instea
5 |+np.cumprod(np.random.rand(5, 5))
6 6 | np.sometrue(np.random.rand(5, 5))
7 7 | np.alltrue(np.random.rand(5, 5))
8 8 |

NPY003.py:6:1: NPY003 [*] `np.sometrue` is deprecated; use `np.any` instead
|
Expand All @@ -78,13 +79,17 @@ NPY003.py:6:1: NPY003 [*] `np.sometrue` is deprecated; use `np.any` instead
6 |-np.sometrue(np.random.rand(5, 5))
6 |+np.any(np.random.rand(5, 5))
7 7 | np.alltrue(np.random.rand(5, 5))
8 8 |
9 9 | from numpy import round_, product, cumproduct, sometrue, alltrue

NPY003.py:7:1: NPY003 [*] `np.alltrue` is deprecated; use `np.all` instead
|
5 | np.cumproduct(np.random.rand(5, 5))
6 | np.sometrue(np.random.rand(5, 5))
7 | np.alltrue(np.random.rand(5, 5))
| ^^^^^^^^^^ NPY003
8 |
9 | from numpy import round_, product, cumproduct, sometrue, alltrue
|
= help: Replace with `np.all`

Expand All @@ -94,5 +99,103 @@ NPY003.py:7:1: NPY003 [*] `np.alltrue` is deprecated; use `np.all` instead
6 6 | np.sometrue(np.random.rand(5, 5))
7 |-np.alltrue(np.random.rand(5, 5))
7 |+np.all(np.random.rand(5, 5))
8 8 |
9 9 | from numpy import round_, product, cumproduct, sometrue, alltrue
10 10 |

NPY003.py:11:1: NPY003 [*] `np.round_` is deprecated; use `np.round` instead
|
9 | from numpy import round_, product, cumproduct, sometrue, alltrue
10 |
11 | round_(np.random.rand(5, 5), 2)
| ^^^^^^ NPY003
12 | product(np.random.rand(5, 5))
13 | cumproduct(np.random.rand(5, 5))
|
= help: Replace with `np.round`

ℹ Suggested fix
8 8 |
9 9 | from numpy import round_, product, cumproduct, sometrue, alltrue
10 10 |
11 |-round_(np.random.rand(5, 5), 2)
11 |+round(np.random.rand(5, 5), 2)
12 12 | product(np.random.rand(5, 5))
13 13 | cumproduct(np.random.rand(5, 5))
14 14 | sometrue(np.random.rand(5, 5))

NPY003.py:12:1: NPY003 [*] `np.product` is deprecated; use `np.prod` instead
|
11 | round_(np.random.rand(5, 5), 2)
12 | product(np.random.rand(5, 5))
| ^^^^^^^ NPY003
13 | cumproduct(np.random.rand(5, 5))
14 | sometrue(np.random.rand(5, 5))
|
= help: Replace with `np.prod`

ℹ Suggested fix
9 9 | from numpy import round_, product, cumproduct, sometrue, alltrue
10 10 |
11 11 | round_(np.random.rand(5, 5), 2)
12 |-product(np.random.rand(5, 5))
12 |+prod(np.random.rand(5, 5))
13 13 | cumproduct(np.random.rand(5, 5))
14 14 | sometrue(np.random.rand(5, 5))
15 15 | alltrue(np.random.rand(5, 5))

NPY003.py:13:1: NPY003 [*] `np.cumproduct` is deprecated; use `np.cumprod` instead
|
11 | round_(np.random.rand(5, 5), 2)
12 | product(np.random.rand(5, 5))
13 | cumproduct(np.random.rand(5, 5))
| ^^^^^^^^^^ NPY003
14 | sometrue(np.random.rand(5, 5))
15 | alltrue(np.random.rand(5, 5))
|
= help: Replace with `np.cumprod`

ℹ Suggested fix
10 10 |
11 11 | round_(np.random.rand(5, 5), 2)
12 12 | product(np.random.rand(5, 5))
13 |-cumproduct(np.random.rand(5, 5))
13 |+cumprod(np.random.rand(5, 5))
14 14 | sometrue(np.random.rand(5, 5))
15 15 | alltrue(np.random.rand(5, 5))

NPY003.py:14:1: NPY003 [*] `np.sometrue` is deprecated; use `np.any` instead
|
12 | product(np.random.rand(5, 5))
13 | cumproduct(np.random.rand(5, 5))
14 | sometrue(np.random.rand(5, 5))
| ^^^^^^^^ NPY003
15 | alltrue(np.random.rand(5, 5))
|
= help: Replace with `np.any`

ℹ Suggested fix
11 11 | round_(np.random.rand(5, 5), 2)
12 12 | product(np.random.rand(5, 5))
13 13 | cumproduct(np.random.rand(5, 5))
14 |-sometrue(np.random.rand(5, 5))
14 |+any(np.random.rand(5, 5))
15 15 | alltrue(np.random.rand(5, 5))

NPY003.py:15:1: NPY003 [*] `np.alltrue` is deprecated; use `np.all` instead
|
13 | cumproduct(np.random.rand(5, 5))
14 | sometrue(np.random.rand(5, 5))
15 | alltrue(np.random.rand(5, 5))
| ^^^^^^^ NPY003
|
= help: Replace with `np.all`

ℹ Suggested fix
12 12 | product(np.random.rand(5, 5))
13 13 | cumproduct(np.random.rand(5, 5))
14 14 | sometrue(np.random.rand(5, 5))
15 |-alltrue(np.random.rand(5, 5))
15 |+all(np.random.rand(5, 5))


34 changes: 34 additions & 0 deletions crates/ruff/src/rules/pyupgrade/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
mod fixes;
mod helpers;
pub(crate) mod rules;
pub mod settings;
pub(crate) mod types;

#[cfg(test)]
Expand All @@ -12,6 +13,7 @@ mod tests {
use test_case::test_case;

use crate::registry::Rule;
use crate::rules::pyupgrade;
use crate::settings::types::PythonVersion;
use crate::test::test_path;
use crate::{assert_messages, settings};
Expand Down Expand Up @@ -85,6 +87,38 @@ mod tests {
Ok(())
}

#[test]
fn future_annotations_keep_runtime_typing_p37() -> Result<()> {
let diagnostics = test_path(
Path::new("pyupgrade/future_annotations.py"),
&settings::Settings {
pyupgrade: pyupgrade::settings::Settings {
keep_runtime_typing: true,
},
target_version: PythonVersion::Py37,
..settings::Settings::for_rule(Rule::NonPEP585Annotation)
},
)?;
assert_messages!(diagnostics);
Ok(())
}

#[test]
fn future_annotations_keep_runtime_typing_p310() -> Result<()> {
let diagnostics = test_path(
Path::new("pyupgrade/future_annotations.py"),
&settings::Settings {
pyupgrade: pyupgrade::settings::Settings {
keep_runtime_typing: true,
},
target_version: PythonVersion::Py310,
..settings::Settings::for_rule(Rule::NonPEP585Annotation)
},
)?;
assert_messages!(diagnostics);
Ok(())
}

#[test]
fn future_annotations_pep_585_p37() -> Result<()> {
let diagnostics = test_path(
Expand Down
76 changes: 76 additions & 0 deletions crates/ruff/src/rules/pyupgrade/settings.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
//! Settings for the `pyupgrade` plugin.

use ruff_macros::{CacheKey, CombineOptions, ConfigurationOptions};
use serde::{Deserialize, Serialize};

#[derive(
Debug, PartialEq, Eq, Serialize, Deserialize, Default, ConfigurationOptions, CombineOptions,
)]
#[serde(
deny_unknown_fields,
rename_all = "kebab-case",
rename = "PyUpgradeOptions"
)]
#[cfg_attr(feature = "schemars", derive(schemars::JsonSchema))]
pub struct Options {
#[option(
default = r#"false"#,
value_type = "bool",
example = r#"
# Preserve types, even if a file imports `from __future__ import annotations`.
keep-runtime-typing = true
"#
)]
/// Whether to avoid PEP 585 (`List[int]` -> `list[int]`) and PEP 604
/// (`Union[str, int]` -> `str | int`) rewrites even if a file imports
/// `from __future__ import annotations`.
///
/// This setting is only applicable when the target Python version is below
/// 3.9 and 3.10 respectively, and is most commonly used when working with
/// libraries like Pydantic and FastAPI, which rely on the ability to parse
/// type annotations at runtime. The use of `from __future__ import annotations`
/// causes Python to treat the type annotations as strings, which typically
/// allows for the use of language features that appear in later Python
/// versions but are not yet supported by the current version (e.g., `str |
/// int`). However, libraries that rely on runtime type annotations will
/// break if the annotations are incompatible with the current Python
/// version.
///
/// For example, while the following is valid Python 3.8 code due to the
/// presence of `from __future__ import annotations`, the use of `str| int`
/// prior to Python 3.10 will cause Pydantic to raise a `TypeError` at
/// runtime:
///
/// ```python
/// from __future__ import annotations
///
/// import pydantic
///
/// class Foo(pydantic.BaseModel):
/// bar: str | int
/// ```
///
///
pub keep_runtime_typing: Option<bool>,
}

#[derive(Debug, Default, CacheKey)]
pub struct Settings {
pub keep_runtime_typing: bool,
}

impl From<Options> for Settings {
fn from(options: Options) -> Self {
Self {
keep_runtime_typing: options.keep_runtime_typing.unwrap_or_default(),
}
}
}

impl From<Settings> for Options {
fn from(settings: Settings) -> Self {
Self {
keep_runtime_typing: Some(settings.keep_runtime_typing),
}
}
}
Loading