Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bpo-40077: Convert _csv module to use PyType_FromSpec #20974

Closed
wants to merge 7 commits into from
Closed

bpo-40077: Convert _csv module to use PyType_FromSpec #20974

wants to merge 7 commits into from

Conversation

corona10
Copy link
Member

@corona10 corona10 commented Jun 19, 2020 •
edited by bedevere-bot
Loading

@corona10 corona10 requested a review from vstinner June 19, 2020 06:14
corona10
corona10 commented Jun 19, 2020
View reviewed changes
Copy link
Member Author

@corona10 corona10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vstinner
Please take a look

I locally tested with _csv module on subinterpreter and module test
And there was no leak.
(But import csv with subinterpreter still has leaks but not related to this PR)

@corona10
Copy link
Member Author

FYI, macOS CI issue is not related to this PR

vstinner
vstinner reviewed Jun 19, 2020
View reviewed changes
Modules/_csv.c Outdated Show resolved Hide resolved
Modules/_csv.c Outdated Show resolved Hide resolved
Modules/_csv.c Show resolved Hide resolved
Modules/_csv.c Outdated Show resolved Hide resolved
@corona10 corona10 requested a review from vstinner June 19, 2020 10:39
corona10
corona10 commented Jun 19, 2020
View reviewed changes
Copy link
Member Author

@corona10 corona10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vstinner

Thanks all of your reviews are applied and no memory leak was found!

vstinner
vstinner requested changes Jun 19, 2020
View reviewed changes
Copy link
Member

@vstinner vstinner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure that it's possible to convert static types to heap types if they have Py_TPFLAGS_BASETYPE, since currently there is no way to retrieve the module from such type.

Modules/_csv.c Show resolved Hide resolved
Modules/_csv.c
@@ -354,25 +376,29 @@ dialect_new(PyTypeObject *type, PyObject *args, PyObject *kwargs)
&strict))
return NULL;

_csvstate *state = PyType_GetModuleState(type);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PyType_GetModuleState() is not safe if the type has Py_TPFLAGS_BASETYPE flag, which is the case here.

Is there a way to get the defining type in tp_new?

Modules/_csv.c
PyErr_Format(_csvstate_global->error_obj, "field larger than field limit (%ld)",
_csvstate_global->field_limit);
PyTypeObject *reader_type = Py_TYPE(self);
_csvstate *state = PyType_GetModuleState(reader_type);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Reader type has Py_TPFLAGS_BASETYPE: PyType_GetModuleState() is unsafe here.

Modules/_csv.c
@@ -961,7 +987,8 @@ csv_reader(PyObject *module, PyObject *args, PyObject *keyword_args)
Py_DECREF(self);
return NULL;
}
self->dialect = (DialectObj *)_call_dialect(dialect, keyword_args);
_csvstate *state = get_csv_state(module);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

state can be moved at the beginning of the function, to avoid calling get_csv_state() twice.

@bedevere-bot
Copy link

When you're done making the requested changes, leave the comment: I have made the requested changes; please review again.

@vstinner
Copy link
Member

I understand that this PR (if merged) would fix https://bugs.python.org/issue14935

@corona10 corona10 mentioned this pull request Sep 27, 2020
Merged
@encukou encukou mentioned this pull request Nov 10, 2020
Merged
@encukou
Copy link
Member

encukou commented Dec 15, 2020

The issue is now closed via #23224.
Thank you for working on it, though!

@encukou encukou closed this Dec 15, 2020
@corona10 corona10 mentioned this pull request Jun 27, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vstinner vstinner vstinner requested changes

Assignees
No one assigned
Labels
awaiting changes
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@corona10 @bedevere-bot @vstinner @encukou @the-knights-who-say-ni