Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-83638: Add sqlite3.Connection.autocommit for PEP 249 compliant behaviour #93823

Merged
merged 82 commits into from
Nov 12, 2022
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
82 commits
Select commit Hold shift + click to select a range
10d3a65
Add sqlite3.Connection.autocommit for PEP 249 compliant behaviour
erlend-aasland Jun 11, 2022
6ca9045
Add basic tests
erlend-aasland Jun 14, 2022
5772676
Add documentation
erlend-aasland Jun 14, 2022
06609f3
Add NEWS
erlend-aasland Jun 14, 2022
8b6b866
Add What's New
erlend-aasland Jun 14, 2022
a604a57
Document DEPRECATED_TRANSACTION_CONTROL
erlend-aasland Jun 14, 2022
cbc1041
Fix setattr behaviour
erlend-aasland Jun 14, 2022
41b898c
Suggest new behaviour starting with Python 3.14
erlend-aasland Jun 15, 2022
b11ebd2
Simplify implicit rollback test
erlend-aasland Jun 15, 2022
d3009d4
Address Alex' review
erlend-aasland Jun 15, 2022
fb98615
Reorder if condition to add emphasis to deprecated behaviour
erlend-aasland Jun 15, 2022
5e4626e
Add context manager tests
erlend-aasland Jun 15, 2022
a8e3b9e
Improve 5e4626ecf47a679ba5fa8846472af16fb325cac5
erlend-aasland Jun 15, 2022
d46e9be
Add executescript tests
erlend-aasland Jun 15, 2022
c2e2f1a
Improve docs
erlend-aasland Jun 15, 2022
3bf3ee1
Add missing versionadded
erlend-aasland Jun 16, 2022
757f70b
Try to further improve the docs
erlend-aasland Jun 16, 2022
a5ef40c
Nano-optimisation
erlend-aasland Jun 16, 2022
ae37fbb
Explicit downcast
erlend-aasland Jun 17, 2022
341e0dd
Clarify execute*, commit, rollback, and close
erlend-aasland Jun 17, 2022
77651c0
Incorporate gh-93890 changes
erlend-aasland Jun 17, 2022
0ac782c
Improve
erlend-aasland Jun 17, 2022
7743f34
Merge branch 'main' into sqlite-autocommit
erlend-aasland Jun 19, 2022
6628e30
commit/rollback wordings
erlend-aasland Jun 19, 2022
060bd3a
Fix merge
erlend-aasland Jun 19, 2022
c96adbf
Doc: reword context manager section
erlend-aasland Jun 20, 2022
a7c7c4c
Autocommit is keyword only
erlend-aasland Jun 24, 2022
f1c5843
Isolation level is not yet deprecated (soon it will be though)
erlend-aasland Jun 24, 2022
491426e
Merge remote-tracking branch 'upstream/main' into sqlite-autocommit
erlend-aasland Jun 24, 2022
05879a9
Merge branch 'main' into sqlite-autocommit
erlend-aasland Jun 27, 2022
97ad734
Use macro as C default
erlend-aasland Jun 27, 2022
bf3a857
Merge branch 'main' into sqlite-autocommit
erlend-aasland Jul 8, 2022
60f7224
Merge branch 'main' into sqlite-autocommit
erlend-aasland Jul 18, 2022
7f645ee
Pass keywords to factory function
erlend-aasland Jul 20, 2022
88fb2b9
Merge branch 'main' into sqlite-autocommit
erlend-aasland Jul 20, 2022
4deed17
Adjust commit/rollback and remove redundant info
erlend-aasland Jul 20, 2022
f3a9e37
Merge branch 'main' into sqlite-autocommit
erlend-aasland Jul 23, 2022
2981a6b
Fix docstring
erlend-aasland Jul 23, 2022
a5b7864
Revert unneeded doc changes
erlend-aasland Jul 23, 2022
fd6659a
Fix autocommit enum
erlend-aasland Jul 23, 2022
52e3736
Sync with main bco. recent doc updates
erlend-aasland Jul 26, 2022
6e92073
Merge branch 'main' into sqlite-autocommit
erlend-aasland Jul 30, 2022
59fbd29
Merge branch 'main' into sqlite-autocommit
erlend-aasland Aug 3, 2022
894fcce
Add autocommit to connect signature
erlend-aasland Aug 3, 2022
fff4307
Merge branch 'main' into sqlite-autocommit
erlend-aasland Aug 8, 2022
4ac0fa0
Address Alex's second review
erlend-aasland Aug 8, 2022
64c6482
Adjust headings
erlend-aasland Aug 8, 2022
e4947d0
Use versionadded iso. versionchanged
erlend-aasland Aug 8, 2022
5981536
Remove some no-ops
erlend-aasland Aug 8, 2022
527e6a3
Three adjustments:
erlend-aasland Aug 8, 2022
697b65a
Address Alex's last round of review
erlend-aasland Aug 15, 2022
4299874
Sync with main bco. recent Argument Clinic changes
erlend-aasland Aug 15, 2022
76122be
Regen all
erlend-aasland Aug 15, 2022
6902738
Explicit logic in set_autocommit
erlend-aasland Aug 15, 2022
c37ebb5
Tweak naming: connection_txn_stmt => connection_exec_stmt
erlend-aasland Aug 15, 2022
2e9083c
Merge branch 'main' into sqlite-autocommit
erlend-aasland Aug 18, 2022
b944a90
Address review: improve wording for connection close()
erlend-aasland Aug 18, 2022
bce921f
Sync with main
erlend-aasland Oct 28, 2022
78e04f0
Fix bad merge
erlend-aasland Oct 28, 2022
bd91e34
Remove implicit commit for cx.autocommit = False
erlend-aasland Oct 28, 2022
9d66daa
Doc: make the autocommit reference more to the point; expand in the e…
erlend-aasland Oct 28, 2022
d4c0583
Address most of CAM's review
erlend-aasland Oct 29, 2022
a3aa9aa
Better naming for the constant; don't mention deprecations (yet)
erlend-aasland Oct 29, 2022
c9b05ff
Fix default role
erlend-aasland Oct 30, 2022
d5e33f1
Make it explicit that isolation_level is ignored if autocommit is Tru…
erlend-aasland Oct 30, 2022
0cf022a
Reflow
erlend-aasland Oct 30, 2022
5009372
Address review: COMPAT_TRANSACTIONAL_CONTROL => LEGACY_TRANSACTION_CO…
erlend-aasland Oct 31, 2022
2c9b117
Address more of CAM's review
erlend-aasland Oct 31, 2022
de3bd4e
Try to address the rest of CAM's review
erlend-aasland Oct 31, 2022
fca9c56
Sync with main
erlend-aasland Nov 6, 2022
7de6bf3
Address more of CAM's remarks
erlend-aasland Nov 6, 2022
e0fa134
Prevent segfault if implicitly rolling back during interpreter shutdown
erlend-aasland Nov 6, 2022
e0bedf6
Update Doc/library/sqlite3.rst
Nov 7, 2022
e33946a
Address last batch of CAM's comments
erlend-aasland Nov 7, 2022
cd15505
Last bit
erlend-aasland Nov 8, 2022
4feefa1
Update Doc/library/sqlite3.rst
Nov 11, 2022
720ac11
Sync with main
erlend-aasland Nov 11, 2022
f7c2ae4
Update Doc/library/sqlite3.rst
Nov 11, 2022
88e1420
Sync with main
erlend-aasland Nov 12, 2022
7b87830
Address review: remove unneeded assert in test_autocommit_setget_invalid
erlend-aasland Nov 12, 2022
8322683
Address review: add test for rollback() if autocommit is True
erlend-aasland Nov 12, 2022
aca1e22
Address review: fixup context manager docs
erlend-aasland Nov 12, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
109 changes: 94 additions & 15 deletions Doc/library/sqlite3.rst
Original file line number Diff line number Diff line change
Expand Up @@ -235,8 +235,14 @@ Module functions and constants
everything until the first ``'['`` for the column name and strip
the preceding space: the column name would simply be "Expiration date".

.. data:: DEPRECATED_TRANSACTION_CONTROL

.. function:: connect(database[, timeout, detect_types, isolation_level, check_same_thread, factory, cached_statements, uri])
Set :attr:`~Connection.autocommit` to this constant to select deprecated
(pre Python 3.12) transaction control.
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved
See :ref:`sqlite3-deprecated-transaction-control` for more information.
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved


.. function:: connect(database[, timeout, detect_types, isolation_level, check_same_thread, factory, cached_statements, uri, autocommit])

Opens a connection to the SQLite database file *database*. By default returns a
:class:`Connection` object, unless a custom *factory* is given.
Expand All @@ -255,6 +261,11 @@ Module functions and constants
For the *isolation_level* parameter, please see the
:attr:`~Connection.isolation_level` property of :class:`Connection` objects.

For the *autocommit* parameter, please see the
:attr:`~Connection.autocommit` property. *autocommit* currently defaults to
:data:`~Connection.DEPRECATED_TRANSACTION_CONTROL`.
The default will change to :const:`False` in Python 3.14.

SQLite natively supports only the types TEXT, INTEGER, REAL, BLOB and NULL. If
you want to use other types you must add support for them yourself. The
*detect_types* parameter and the using custom **converters** registered with the
Expand Down Expand Up @@ -318,6 +329,10 @@ Module functions and constants
.. versionchanged:: 3.10
Added the ``sqlite3.connect/handle`` auditing event.

.. versionchanged:: 3.12
Added the *autocommit* parameter.
Deprecated the *isolation_level* parameter.


.. function:: register_converter(typename, callable)

Expand Down Expand Up @@ -385,6 +400,24 @@ Connection Objects

An SQLite database connection has the following attributes and methods:

.. attribute:: autocommit

Get or set the :pep:`249` transaction behaviour.
*autocommit* has tree allowed values:
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved

* :const:`False`: PEP 249 compliant transaction behaviour,
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved
implying that a transaction is always open.
Use :meth:`commit` and :meth:`rollback` to close transactions.
Closing a transaction immediately opens a new one.
This will be the default value of *autocommit* in Python 3.14.

* :const:`True`: Use SQLite's autocommit behaviour.
You are also free to :meth:`execute` custom transaction statements.

* :data:`DEPRECATED_TRANSACTION_CONTROL`: Pre Python 3.12 compliant
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved
transaction control. See :attr:`isolation_level`.
This is currently the default value of *autocommit*.

.. attribute:: isolation_level

Get or set the current default isolation level. :const:`None` for autocommit mode or
Expand Down Expand Up @@ -1365,41 +1398,83 @@ timestamp converter.

.. _sqlite3-controlling-transactions:

Controlling Transactions
------------------------
Controlling Transactions Using the Autocommit Property
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved
------------------------------------------------------

The underlying ``sqlite3`` library operates in ``autocommit`` mode by default,
but the Python :mod:`sqlite3` module by default does not.

``autocommit`` mode means that statements that modify the database take effect
Use the :attr:`~Connection.autocommit` property to select transaction mode.
This attribute can also be set when :meth:`connecting <~Connection.connect>`.

Currently, *autocommit* defaults to :const:`DEPRECATED_TRANSACTION_CONTROL`,
which means transaction control is selected using the
:attr:`~Connection.isolation_level` property.
See :ref:`sqlite3-deprecated-transaction-control` for more information.

Starting with Python 3.14, *autocommit* will default to :const:`False`,
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved
which will imply :pep:`249` compliant transaction control. This means:
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved

* A transaction is always open.
* Commit transactions using :meth:`~Connection.commit`.
* Roll back transactions using :meth:`~Connection.rollback`.
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved
* Commit and rollback will open a new transaction immediately after execution.
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved
* An implicit rollback is performed if the database is closed with pending
changes.

Set *autocommit* to :const:`True` to enable SQLite's autocommit mode.
This mode is equal to setting :attr:`~Connection.isolation_level` to
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved
:const:`None`.

``autocommit=True`` means that statements that modify the database take effect
immediately. A ``BEGIN`` or ``SAVEPOINT`` statement disables ``autocommit``
mode, and a ``COMMIT``, a ``ROLLBACK``, or a ``RELEASE`` that ends the
outermost transaction, turns ``autocommit`` mode back on.

The Python :mod:`sqlite3` module by default issues a ``BEGIN`` statement
implicitly before a Data Modification Language (DML) statement (i.e.
``INSERT``/``UPDATE``/``DELETE``/``REPLACE``).

You can control which kind of ``BEGIN`` statements :mod:`sqlite3` implicitly
.. _sqlite3-deprecated-transaction-control:

Controlling Transactions Using the Isolation Level Property
-----------------------------------------------------------

.. note::

The recommended way of controlling transactions, is via the
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved
:attr:`~Connection.autocommit` parameter.

If :attr:`~Connection.autocommit` is set to
:data:`DEPRECATED_TRANSACTION_CONTROL`, transaction control is selected using
the :attr:`~Connection.isolation_level` parameter.

``isolation_level`` defaults to the empty string: ``""``. This implies that
the Python :mod:`sqlite3` module issues a ``BEGIN`` statement
implicitly before a Data Modification Language (DML) statement
(``INSERT``, ``UPDATE``, ``DELETE``, and ``REPLACE``).

You can control the kind of ``BEGIN`` statement ``sqlite3`` implicitly
executes via the *isolation_level* parameter to the :func:`connect`
call, or via the :attr:`isolation_level` property of connections.
If you specify no *isolation_level*, a plain ``BEGIN`` is used, which is
equivalent to specifying ``DEFERRED``. Other possible values are ``IMMEDIATE``
and ``EXCLUSIVE``.
call, or via the :attr:`isolation_level` property of connection objects.
By default, ``BEGIN`` is used, which is equivalent to
``isolation_level="DEFERRED"``.
Other possible values are ``"IMMEDIATE"`` and ``"EXCLUSIVE"``.

You can disable the :mod:`sqlite3` module's implicit transaction management by
Disable the :mod:`sqlite3` module's implicit transaction control by
setting :attr:`isolation_level` to ``None``. This will leave the underlying
``sqlite3`` library operating in ``autocommit`` mode. You can then completely
control the transaction state by explicitly issuing ``BEGIN``, ``ROLLBACK``,
control transactions by explicitly executing ``BEGIN``, ``ROLLBACK``,
``SAVEPOINT``, and ``RELEASE`` statements in your code.

Note that :meth:`~Cursor.executescript` disregards
:attr:`isolation_level`; any transaction control must be added explicitly.

.. versionchanged:: 3.6
:mod:`sqlite3` used to implicitly commit an open transaction before DDL
``sqlite3`` used to implicitly commit an open transaction before DDL
statements. This is no longer the case.

.. versionchanged: 3.12
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved
The recommended way of controlling transactions is now via the
:attr:`~Connection.autocommit` parameter.
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved


Using :mod:`sqlite3` efficiently
--------------------------------
Expand Down Expand Up @@ -1441,6 +1516,10 @@ committed:

.. literalinclude:: ../includes/sqlite3/ctx_manager.py

.. note::

The context manager does not implicitly begin a new transaction.


.. rubric:: Footnotes

Expand Down
11 changes: 11 additions & 0 deletions Doc/whatsnew/3.12.rst
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,14 @@ os
(Contributed by Kumar Aditya in :gh:`93312`.)


sqlite3
-------

* Add :attr:`~sqlite3.Connection.autocommit` parameter for :pep:`249` compliant
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved
transaction handling.
(Contributed by Erlend E. Aasland in :gh:`83638`.)


Optimizations
=============

Expand All @@ -118,6 +126,9 @@ CPython bytecode changes
Deprecated
==========

* The :attr:`sqlite3.Connection.isolation_level` parameter is deprecated.
Please use the new :attr:`sqlite3.Connection.autocommit` parameter instead.
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved


Pending Removal in Python 3.13
==============================
Expand Down
98 changes: 98 additions & 0 deletions Lib/test/test_sqlite3/test_transactions.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@

import os, unittest
import sqlite3 as sqlite
from contextlib import contextmanager

from test.support import LOOPBACK_TIMEOUT
from test.support.os_helper import TESTFN, unlink
Expand Down Expand Up @@ -327,5 +328,102 @@ def test_isolation_level_none(self):
self.assertEqual(self.traced, [self.QUERY])


class AutocommitPEP249(unittest.TestCase):
"""Test PEP-249 compliant autocommit behaviour."""
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved
@contextmanager
def check_stmt_trace(self, cx, expected, reset=True):
try:
traced = []
cx.set_trace_callback(lambda stmt: traced.append(stmt))
yield
finally:
self.assertEqual(traced, expected)
if reset:
cx.set_trace_callback(None)

def test_autocommit_default(self):
with memory_database() as cx:
self.assertEqual(cx.autocommit,
sqlite.DEPRECATED_TRANSACTION_CONTROL)

def test_autocommit_setget(self):
dataset = (
True,
False,
sqlite.DEPRECATED_TRANSACTION_CONTROL,
)
for mode in dataset:
with self.subTest(mode=mode):
with memory_database(autocommit=mode) as cx:
self.assertEqual(cx.autocommit, mode)
with memory_database() as cx:
cx.autocommit = mode
self.assertEqual(cx.autocommit, mode)

def test_autocommit_setget_invalid(self):
msg = "autocommit must be True, False, or.*DEPRECATED"
for mode in "a", 12, (), None:
with self.subTest(mode=mode):
with self.assertRaisesRegex(ValueError, msg):
with memory_database(autocommit=mode) as cx:
self.assertEqual(cx.autocommit, mode)
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved

def test_autocommit_disabled(self):
expected = [
"SELECT 1",
"COMMIT",
"BEGIN",
"ROLLBACK",
"BEGIN",
]
with memory_database(autocommit=False) as cx:
self.assertTrue(cx.in_transaction)
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved
with self.check_stmt_trace(cx, expected):
cx.execute("SELECT 1")
cx.commit()
cx.rollback()

def test_autocommit_disabled_implicit_rollback(self):
expected = ["ROLLBACK"]
with memory_database(autocommit=False) as cx:
self.assertTrue(cx.in_transaction)
with self.check_stmt_trace(cx, expected, reset=False):
cx.close()

def test_autocommit_enabled(self):
expected = ["SELECT 1"]
with memory_database(autocommit=True) as cx:
self.assertFalse(cx.in_transaction)
with self.check_stmt_trace(cx, expected):
cx.execute("SELECT 1")
cx.commit() # expect this to pass silently
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved
self.assertFalse(cx.in_transaction)

def test_autocommit_disabled_then_enabled(self):
expected = ["COMMIT"]
with memory_database(autocommit=False) as cx:
self.assertTrue(cx.in_transaction)
with self.check_stmt_trace(cx, expected):
cx.autocommit = True # should commit
self.assertFalse(cx.in_transaction)

def test_autocommit_enabled_then_disabled(self):
expected = ["BEGIN"]
with memory_database(autocommit=True) as cx:
self.assertFalse(cx.in_transaction)
with self.check_stmt_trace(cx, expected):
cx.autocommit = False # should begin
self.assertTrue(cx.in_transaction)

erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved
def test_autocommit_explicit_then_disabled(self):
expected = ["BEGIN DEFERRED", "COMMIT", "BEGIN"]
with memory_database(autocommit=True) as cx:
self.assertFalse(cx.in_transaction)
with self.check_stmt_trace(cx, expected):
cx.execute("BEGIN DEFERRED")
cx.autocommit = False # should commit, then begin
self.assertTrue(cx.in_transaction)


if __name__ == "__main__":
unittest.main()
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Add :attr:`sqlite3.Connection.autocommit` for :pep:`249` compliant
transaction control. Patch by Erlend E. Aasland.
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved
26 changes: 18 additions & 8 deletions Modules/_sqlite/clinic/connection.c.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading