TypeError: argument 2 must be a connection, cursor or None when migrate to django 3.1.1

[aleksandr-hilko]

New version of django (3.1.1) was released recently. After migrating a project to this version when psycopg2 is trying to register extensions like JSON raises such an error:

TypeError: argument 2 must be a connection, cursor or None when migrate to django 3.1.1. This is not reproduced in django

This issue is not reproduced in django 3.1.

Components that are used in my project are: python==3.6, django==3.1.1 , psycopg2==2.8.6

Here is a stack trace of an error:

Traceback (most recent call last):
  File "/app/manage.py", line 30, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py", line 395, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.6/site-packages/django/core/management/base.py", line 330, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.6/site-packages/django/core/management/base.py", line 371, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.6/site-packages/django/core/management/base.py", line 85, in wrapped
    res = handle_func(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/django/core/management/commands/migrate.py", line 90, in handle
    connection.prepare_database()
  File "/usr/local/lib/python3.6/site-packages/django/contrib/gis/db/backends/postgis/base.py", line 25, in prepare_database
    with self.cursor() as cursor:
  File "/usr/local/lib/python3.6/site-packages/aws_xray_sdk/ext/django/db.py", line 81, in cursor
    original_cursor = getattr(self, attr)(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 259, in cursor
    return self._cursor()
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 235, in _cursor
    self.ensure_connection()
  File "/usr/local/lib/python3.6/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 219, in ensure_connection
    self.connect()
  File "/usr/local/lib/python3.6/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 200, in connect
    self.connection = self.get_new_connection(conn_params)
  File "/usr/local/lib/python3.6/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/postgresql/base.py", line 206, in get_new_connection
    psycopg2.extras.register_default_jsonb(conn_or_curs=connection, loads=lambda x: x)
  File "/usr/local/lib/python3.6/site-packages/psycopg2/_json.py", line 156, in register_default_jsonb
    loads=loads, oid=JSONB_OID, array_oid=JSONBARRAY_OID, name='jsonb')
  File "/usr/local/lib/python3.6/site-packages/psycopg2/_json.py", line 125, in register_json
    register_type(JSON, not globally and conn_or_curs or None)
TypeError: argument 2 must be a connection, cursor or None

[bhautikpip]

Hi,

Looked at the Django release notes of v3.1.1 couldn't find anything that should cause this issue. Would you be able post small repro code ?

[ezet]

This is the same issue as mentioned in #245 , and I'm also unable to use x-ray because of this

[ezet]

Further investigation reveals that this change is the culprit:
https://github.com/django/django/pull/13358/files#diff-56374f35499bd669398d3100ef7fcea371a8e9e38f92dbd3569d75b8cea2ad93R206

Issue tracker: https://code.djangoproject.com/ticket/31956

[srprash]

I was able to reproduce the issue with Django 3.1.4 and Psycopg2 2.8.6. Will do some more digging.
@ezet You mentioned that you're testing a solution, can you provide some details?

[ezet]

I have monekypatched the register_type function in manage.py as well as asgi.py and wsgi.py (basically all entrypoints).

Seems to work fine so far.

Below is a basic example for patching this

def monkeypatch_psycopg2_register_type():
    import psycopg2._json

    f = psycopg2._json.register_type

    def func(obj, conn_or_curs):
        from aws_xray_sdk.ext.dbapi2 import XRayTracedConn

        if type(conn_or_curs) == XRayTracedConn:
            conn_or_curs = conn_or_curs.__wrapped__
        return f(obj, conn_or_curs)

    psycopg2._json.register_type = func

[mjhanke]

Any updates on this? I'd hate to have to choose between installing Django 3.1 with no maintenance releases and this plugin.

Edit: workaround is to patch() individual modules instead of patch_all() which would patch psycopg2

[srprash]

Hi @mjhanke
Apologies for falling behind on this issue. We will try to prioritize the investigation and will post an update.
Thanks for mentioning the workaround. :)

[codyfazio]

@srprash Any updates on this? We're also in need of upgrading the Django version in one our projects so hoping a fix is in the works here.

[isaacphi]

@srprash are there any updates? It's something I'm running into as well.

[TeaDove]

And updates?

[demiurg]

any updates? ;]

[srprash]

Hello all. Sincere apologies for not providing any update on this issue.
We are now actively investigating this issue on our end and will provide an update as soon as possible.

Thanks for your patience.

[srprash]

Note: We have seen a similar issue in the past with the register_type API which prompted this sort of fix.
#95

[srprash]

The fix has been released in v2.11.0