Pull in pydevd sys.monitoring support (#1680)

This is pulling in @fabioz's latest changes for sys.monitoring and then fixing up any issues found with the debugpy tests.

A lot of the changes were made by Fabio since the latest pull from pydevd, so I also created this PR to compare what changes I made to Fabio's baseline:
rchiodo/PyDev.Debugger#1

Meaning you really only need to look at that other PR to see what changes I made. The rest of the changes here are from Fabio or ruff doing reformating.

After this goes through, we should have sys.monitoring support in debugpy. We can decide later if we want to implement our own support as @int19h started.

Fixes #1496

.gitignore

@@ -85,7 +85,7 @@ celerybeat-schedule
 *.sage.py
 
 # virtualenv
-.venv
+.venv*
 venv/
 ENV/
 

CONTRIBUTING.md

@@ -93,6 +93,33 @@ The tests are run concurrently, and the default number of workers is 8. You can
 
 While tox is the recommended way to run the test suite, pytest can also be invoked directly from the root of the repository. This requires packages in tests/requirements.txt to be installed first.
 
+#### Keeping logs on test success
+
+There's an internal setting `debugpy_log_passed` that if set to true will not erase the logs after a successful test run. Just search for this in the code and remove the code that deletes the logs on success.
+
+#### Adding logging
+
+Using `pydevd_log.debug` you can add logging just about anywhere in the pydevd code. However this code won't be called if CYTHON support is enabled without recreating the Cython output. To temporarily disable CYTHON support, look for `CYTHON_SUPPORTED` and make sure it's set to False
+
+## Updating pydevd
+
+Pydevd (at src/debugpy/_vendored/pydevd) is a copy of https://github.com/fabioz/PyDev.Debugger. We do not use a git submodule but instead just copy the source. 
+
+In order to update the source, you would:
+- Sync to the appropriate commit in a pydevd repo
+- Diff this against the src/debugpy/_vendored/pydevd folder, being careful to not remove the edits made in the debugpy version
+- Run our tests
+- Make any fixes to get the tests to pass (see logging on how to debug)
+
+You might need to regenerate the Cython modules after any changes. This can be done by:
+
+- Install Python latest (3.12 as of this writing)
+- pip install cython, django>=1.9, setuptools>=0.9, wheel>0.21, twine
+- On a windows machine:
+  - set FORCE_PYDEVD_VC_VARS=C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\VC\Auxiliary\Build\vcvars64.bat
+  - set PYDEVD_FORCE_BUILD_ALL=True
+  - in the pydevd folder: python .\build_tools\build.py
+
 ## Using modified debugpy in Visual Studio Code
 To test integration between debugpy and Visual Studio Code, the latter can be directed to use a custom version of debugpy in lieu of the one bundled with the Python extension. This is done by specifying `"debugAdapterPath"` in `launch.json` - it must point at the root directory of the *package*, which is `src/debugpy` inside the repository:
 

src/debugpy/_vendored/pydevd/.gitignore

@@ -37,3 +37,4 @@ snippet.py
 build/*
 .pytest_cache
 /.mypy_cache/
+.DS_Store

src/debugpy/_vendored/pydevd/README.md

@@ -1,24 +1,72 @@
-PyDev.Debugger
-==============
+# PyDev.Debugger
+
+
+## New
+
+Latest `3.x` version: the PyDev debugger now supports `sys.monitoring` which enables
+really fast tracing on `Python 3.12` (so, if speed is an issue, make sure you upgrade).
+
+## Important
+
+https://github.com/fabioz/PyDev.Debugger is the main repository
+for `pydevd` and the latest versions can always be used directly in:
+
+- [PyDev for Eclipse](http://pydev.org): Enables the usage of `pydevd` in Eclipse (Open Source).
+
+- [Python Debugger (PyDev) for VSCode](https://marketplace.visualstudio.com/items?itemName=fabioz.vscode-pydev-python-debugger): Enables
+    the usage of `pydevd` in VSCode (note that while `pydevd` itself is open source, this extension is commercial 
+    and helps in the development of the Open Source version. It has a free trial and can be used by acquiring a license for 
+    `PyDev for VSCode` at: https://www.pydev.org/vscode/index.html).
+
+    Note that the `Python Debugger (PyDev) for VSCode` may be used as a standalane extension for debugging `Python` by
+    creating the proper configuration in a `launch.json` and launching it.
+
+    Alternatively, [PyDev for VSCode](https://marketplace.visualstudio.com/items?itemName=fabioz.vscode-pydev)
+    leverages it to offer additional features such as debugging of test cases.
+
+## History / Support
+
+The `PyDev Debugger` (`pydevd` for short) is a **Python debugger** which historically was created to
+work with `PyDev` (in Eclipse).
+
+Over the years (as it's open source -- EPL) it was adopted by other IDEs/companies
+(so, it was integrated into PyCharm and VSCode Python through `debugpy`, which also bundles `pydevd`).
+
+Note that although it was adopted by other IDEs (and over the years companies of other
+commercial IDEs did provide backing), by far most of the work was done without any
+external backing and the ongoing work on the project relies on community support.
+
+So, if you like using it, please consider becoming a backer of the project (this is
+done through the `PyDev` umbrella, so please see https://www.pydev.org/about.html
+for how to contribute to the project).
+
+
+## Source code / using
 
 The sources for the PyDev.Debugger may be seen at:
 
 https://github.com/fabioz/PyDev.Debugger
 
 In general, the debugger backend should **NOT** be installed separately if you're using an IDE which already
-bundles it (such as PyDev, PyCharm or bundled through debugpy, which is the debug adapter used in 
-VSCode Python and Visual Studio Python).
+bundles it (such as [PyDev for Eclipse](http://pydev.org), [Python Debugger (PyDev) for VSCode](https://marketplace.visualstudio.com/items?itemName=fabioz.vscode-pydev-python-debugger), 
+PyCharm or the Microsoft Python VSCode Extension, which uses `debugpy`, which is another debug adapter bundling `pydevd` to be used in the Microsoft 
+VSCode Python Extension and Visual Studio Python).
 
 It is however available in PyPi so that it can be installed for doing remote debugging with `pip` -- so, when
 debugging a process which runs in another machine, it's possible to `pip install pydevd` and in the code use
-`pydevd.settrace(host='10.1.1.1')` to connect the debugger backend to the debugger UI running in the IDE
+`pydevd.settrace(host="10.1.1.1")` (in PyDev) or `pydevd.settrace(host="10.1.1.1", protocol="dap")` (in PyDev for VSCode)
+to connect the debugger backend to the debugger UI running in the IDE
 (whereas previously the sources had to be manually copied from the IDE installation).
 
-`pydevd` is compatible with Python 3.6 onwards.
+For instructions on how to `Remote Debug` with `PyDev`, see: https://www.pydev.org/manual_adv_remote_debugger.html
 
-For `Python 2` please keep using `pydevd 2.8.0`.
+For instructions on how to `Remote Debug` with `PyDev for VSCode`, see: https://marketplace.visualstudio.com/items?itemName=fabioz.vscode-pydev-python-debugger
+
+`pydevd` is compatible with Python 3.8 onwards and is tested both with CPython as well as PyPy.
 
-`pydevd` is tested both with CPython as well as PyPy.
+For `Python 3.3 to 3.7` please keep using `pydevd 2.10.0`.
+
+For `Python 2` please keep using `pydevd 2.8.0`.
 
 Recent versions contain speedup modules using Cython, which are generated with a few changes in the regular files
 to `cythonize` the files. To update and compile the cython sources (and generate some other auto-generated files),