Use stricter serializer for unions of simple types

[alexdrydew]

Change Summary

This PR forces simple serializer to fail in case of strict extra.check mode if current type is a subtype of expected type (but not the type itself). With this union serializer is given a chance to find a better serializer during strict pass.

Related issue number

This fixes Union[bool, int] case of pydantic/pydantic#6830 (which was reported in pydantic/pydantic#7294). Ambiguity of union serializer in case of arbitrary types (e.g. with is-instance schema) is not fixed.

Checklist

• Unit tests for the changes exist
• Documentation reflects the changes where applicable
• Pydantic tests pass with this pydantic-core (except for expected changes)
• My PR is ready to review, please add a comment including the phrase "please review" to assign reviewers

Selected Reviewer: @samuelcolvin

[codecov]

Codecov Report

Merging #1132 (16370b4) into main (a1b934f) will increase coverage by 0.06%.
Report is 1 commits behind head on main.
The diff coverage is 92.10%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1132      +/-   ##
==========================================
+ Coverage   89.88%   89.95%   +0.06%     
==========================================
  Files         106      106              
  Lines       16436    16438       +2     
  Branches       35       36       +1     
==========================================
+ Hits        14773    14786      +13     
+ Misses       1656     1645      -11     
  Partials        7        7              

Files	Coverage Δ
python/pydantic_core/core_schema.py	94.76% <100.00%> (-0.01%)	⬇️
src/serializers/type_serializers/simple.rs	96.33% <100.00%> (+0.17%)	⬆️
src/validators/mod.rs	95.37% <100.00%> (-0.02%)	⬇️
src/serializers/shared.rs	79.66% <0.00%> (-0.68%)	⬇️
src/serializers/type_serializers/tuple.rs	96.17% <96.10%> (+2.92%)	⬆️
src/validators/union.rs	89.41% <0.00%> (+1.80%)	⬆️
src/validators/tuple.rs	94.07% <92.92%> (+0.69%)	⬆️

... and 3 files with indirect coverage changes

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4df7624...16370b4. Read the comment docs.

[codspeed-hq]

CodSpeed Performance Report

Merging #1132 will degrade performances by 18.75%

_{Comparing alexdrydew:pick_correct_serializer_for_simple_unions (16370b4) with main (4df7624)}

Summary

❌ 1 regressions
✅ 139 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

	Benchmark	main	alexdrydew:pick_correct_serializer_for_simple_unions	Change
❌	test_core_str	31.4 µs	38.7 µs	-18.75%

[alexdrydew]

please review

[alexdrydew]

@samuelcolvin
I am not sure how should I handle failing test-pydantic-integration case: there is a test case in pydantic repository which checks for the behavior that will be changed with this PR:

class Model(BaseModel):
    value: Union[int, Literal['False']]
    
assert Model(value=False).model_dump() == {'value': 0}    

For now I've made a PR that just removes conflicting test cases: pydantic/pydantic#8436

[samuelcolvin]

Don't worry about that, we can fix once pydantic-core is released.

I'll look at this asap, might be after Christmas.

[sydney-runkle]

Also potentially related: pydantic/pydantic#7946

[sydney-runkle]

Also potentially related: pydantic/pydantic#7946

[sydney-runkle]

@alexdrydew,

Thanks for your contribution here. This looks great. Thanks for adding some comprehensive tests and a great description of the changes overall.

I'd like to get this into the next pydantic-core minor release, then we can modify the tests on the pydantic end as well.

Again, great work!

[davidhewitt]

Thanks for the PR! I agree this is correct, but I think a little extra code & test is needed for an edge case...

[davidhewitt]

I think by introducing this you need to implement retry_with_lax_check for these serializers.

I think a test which attempts to serialize an int subclass in a Union[bool, int] would show this to be necessary.

[alexdrydew]

You are right, such test fails due to falling back to infer_to_python. I've implemented retry_with_lax_check which is always false for bool type (as there should be no bool subtypes in python) and true for int type

[davidhewitt]

Thanks, this looks solid to me now 👍

[aelamran]

Hi pydantic team, in what version will this fix be released? and when?
Thank you.

[sydney-runkle]

@alexdrydew,

Great question! This should be out now with pydantic 2.6.0b1 and pydantic-core 2.16.0 :). We'll release 2.6.0 early next week!

[aelamran]

@sydney-runkle was this released? I believe I still have the conversion to int for booleans within a union Type for pydantic 2.6.1.

[sydney-runkle]

Yep, this is released!

[aelamran]

Yep, this is released!

In what version 🙏 ?

[sydney-runkle]

Should be released in Pydantic v2.7 and above!