Add docs and migration guide

CHANGELOG.md

@@ -1,5 +1,6 @@
 # Changelog
-All notable changes to this project will be documented in this file.
+All notable changes to this project will be documented in this file. For help with updating to new
+PyO3 versions, please see the [migration guide](https://pyo3.rs/master/migration.html).
 
 The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
 and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
@@ -17,7 +18,10 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
 - Add `#[derive(FromPyObject)]` macro for enums and structs. [#1065](https://github.com/PyO3/pyo3/pull/1065)
 
 ### Changed
-- Exception types have been renamed from e.g. `RuntimeError` to `PyRuntimeError`, and are now only accessible by `&T` or `Py<T>` similar to other Python-native types. The old names continue to exist but are deprecated. [#1024](https://github.com/PyO3/pyo3/pull/1024)
+- Exception types have been renamed from e.g. `RuntimeError` to `PyRuntimeError`, and are now accessible by `&T` or `Py<T>` similar to other Python-native types. The old names continue to exist but are deprecated. [#1024](https://github.com/PyO3/pyo3/pull/1024) [#1115](https://github.com/PyO3/pyo3/pull/1115)
+  - Rename `PyException::py_err()` to `PyException::new_err()`.
+  - Rename `PyUnicodeDecodeErr::new_err()` to `PyUnicodeDecodeErr::new()`.
+  - Remove `PyStopIteration::stop_iteration()`.
 - Correct FFI definitions `Py_SetProgramName` and `Py_SetPythonHome` to take `*const` argument instead of `*mut`. [#1021](https://github.com/PyO3/pyo3/pull/1021)
 - Rename `PyString::to_string` to `to_str`, change return type `Cow<str>` to `&str`. [#1023](https://github.com/PyO3/pyo3/pull/1023)
 - Correct FFI definition `_PyLong_AsByteArray` `*mut c_uchar` argument instead of `*const c_uchar`. [#1029](https://github.com/PyO3/pyo3/pull/1029)
@@ -28,17 +32,12 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
 - `IntoPy` is no longer implied by `FromPy`. [#1063](https://github.com/PyO3/pyo3/pull/1063)
 - `PyObject` is now just a type alias for `Py<PyAny>`. [#1063](https://github.com/PyO3/pyo3/pull/1063)
 - Rework PyErr to be compatible with the `std::error::Error` trait: [#1067](https://github.com/PyO3/pyo3/pull/1067) [#1115](https://github.com/PyO3/pyo3/pull/1115)
-  - Implement `Display`, `Error`, `Send` and `Sync` for `PyErr`.
-  - `PyErrArguments` now requires `Send + Sync`.
+  - Implement `Display`, `Error`, `Send` and `Sync` for `PyErr` and `PyErrArguments`.
   - Add `PyErr::instance()` which returns `&PyBaseException`.
   - `PyErr`'s fields are now an implementation detail. The equivalent values can be accessed with `PyErr::ptype()`, `PyErr::pvalue()` and `PyErr::ptraceback()`.
   - Change `PyErr::print()` and `PyErr::print_and_set_sys_last_vars()` to take `&self` instead of `self`.
   - Remove `PyErrValue`, `PyErr::from_value`, `PyErr::into_normalized()`, and `PyErr::normalize()`.
-  - Remove `PyException::into()`.
-  - Remove  `Into<PyResult<T>>` for `PyErr` and `PyException`.
-  - Rename `PyException::py_err()` to `PyException::new_err()`.
-  - Rename `PyUnicodeDecodeErr::new_err()` to `PyUnicodeDecodeErr::new()`.
-  - Remove `PyStopIteration::stop_iteration()`.
+  - Remove `PyException::into()` and `Into<PyResult<T>>` for `PyErr` and `PyException`.
 - Change `#[pyproto]` to return NotImplemented for operators for which Python can try a reversed operation. #[1072](https://github.com/PyO3/pyo3/pull/1072)
 - `PyModule::add` now uses `IntoPy<PyObject>` instead of `ToPyObject`. #[1124](https://github.com/PyO3/pyo3/pull/1124)
 

guide/src/exception.md

@@ -115,38 +115,29 @@ fn main() {
 [`Python::is_instance`] calls the underlying [`PyType::is_instance`](https://docs.rs/pyo3/latest/pyo3/types/struct.PyType.html#method.is_instance)
 method to do the actual work.
 
-To check the type of an exception, you can simply do:
+To check the type of an exception, you can similarly do:
 
 ```rust
 # use pyo3::exceptions::PyTypeError;
 # use pyo3::prelude::*;
-# fn main() {
 # let gil = Python::acquire_gil();
 # let py = gil.python();
 # let err = PyTypeError::new_err(());
 err.is_instance::<PyTypeError>(py);
-# }
 ```
 
 ## Handling Rust errors
 
-The vast majority of operations in this library will return [`PyResult<T>`](https://docs.rs/pyo3/latest/pyo3/prelude/type.PyResult.html),
+The vast majority of operations in this library will return
+[`PyResult<T>`](https://docs.rs/pyo3/latest/pyo3/prelude/type.PyResult.html),
 which is an alias for the type `Result<T, PyErr>`.
 
-A [`PyErr`] represents a Python exception.
-Errors within the PyO3 library are also exposed as Python exceptions.
-
-The PyO3 library handles Python exceptions in two stages. During the first stage, a [`PyErr`] instance is
-created. At this stage, holding Python's GIL is not required. During the second stage, an actual Python
-exception instance is created and set active in the Python interpreter.
+A [`PyErr`] represents a Python exception. Errors within the PyO3 library are also exposed as
+Python exceptions.
 
-In simple cases, for custom errors adding an implementation of `std::convert::From<T>` trait
-for this custom error is enough. `PyErr::new` accepts an argument in the form
-of `ToPyObject + 'static`. If the `'static` constraint can not be satisfied or
-more complex arguments are required, the
-[`PyErrArguments`](https://docs.rs/pyo3/latest/pyo3/trait.PyErrArguments.html)
-trait can be implemented. In that case, actual exception argument creation is delayed
-until a `Python` object is available.
+If your code has a custom error type e.g. `MyError`, adding an implementation of
+`std::convert::From<MyError> for PyErr` is usually enough. PyO3 will then automatically convert
+your error to a Python exception when needed.
 
 ```rust
 # use pyo3::{PyErr, PyResult};
@@ -174,7 +165,8 @@ impl std::convert::From<CustomIOError> for PyErr {
     }
 }
 
-fn connect(s: String) -> PyResult<bool> {
+#[pyfunction]
+fn connect(s: String) -> Result<bool, CustomIOError> {
     bind("127.0.0.1:80")?;
     Ok(true)
 }
@@ -195,6 +187,10 @@ fn parse_int(s: String) -> PyResult<usize> {
 
 The code snippet above will raise a `ValueError` in Python if `String::parse()` returns an error.
 
+If lazy construction of the Python exception instance is desired, the
+[`PyErrArguments`](https://docs.rs/pyo3/latest/pyo3/trait.PyErrArguments.html)
+trait can be implemented. In that case, actual exception argument creation is delayed
+until the `PyErr` is needed.
 
 ## Using exceptions defined in Python code
 

guide/src/migration.md

@@ -5,13 +5,91 @@ For a detailed list of all changes, see [CHANGELOG.md](https://github.com/PyO3/p
 
 ## from 0.11.* to 0.12
 
+### `PyErr` has been reworked
+
+In PyO3 `0.12` the `PyErr` type has been re-implemented to be significantly more compatible with
+the standard Rust error handling ecosystem. Specificially `PyErr` now implements
+`Error + Send + Sync`, which are the standard traits used for error types.
+
+While this has necessitated the removal of a number of APIs, the resulting `PyErr` type should now
+be much more easier to work with. The following sections list the changes in detail and how to
+migrate to the new APIs.
+
+#### `PyErr::new` and `PyErr::from_type` now require `Send + Sync` for their argument
+
+For most uses no change will be needed. If you are trying to construct `PyErr` from a value that is
+not `Send + Sync`, you will need to first create the Python object and then use
+`PyErr::from_instance`.
+
+Similarly, any types which implemented `PyErrArguments` will now need to be `Send + Sync`.
+
+#### `PyErr`'s contents are now private
+
+Making the `PyErr` internals private makes it possible for PyO3 to implement optimizations safely
+without needing to worry about breaking downstream code.
+
+If you needed to access the contents of the `PyErr`, you can use the new methods `PyErr::ptype()`,
+`PyErr::pvalue()` and `PyErr::ptraceback()` to safely do so.
+
+#### `PyErrValue` and `PyErr::from_value` have been removed
+
+As these were part the internals of `PyErr` which have been reworked, these APIs no longer exist.
+
+If you used this API, it is recommended to use `PyException::new_err` (see the next section on
+Exception types).
+
+#### `Into<PyResult<T>>` for `PyErr` has been removed
+
+This implementation was not idiomatic Rust. Just construct the `Result::Err` variant directly.
+
+Before:
+```rust,ignore
+let result: PyResult<()> = PyErr::new::<TypeError, _>("error message").into();
+```
+
+After (also using the new reworked exception types; see the following section):
+```rust
+# use pyo3::{PyErr, PyResult, exceptions::PyTypeError};
+let result: PyResult<()> = Err(PyTypeError::new_err("error message"));
+```
+
+### Exception types have been reworked
+
+Previously exception types were zero-sized marker types purely used to construct `PyErr`. In PyO3
+0.12, these types have been replaced with real definitions similar to `PyAny`, `PyDict` etc. This
+makes it is possible to interact with Python exception objects.
+
+The new types also have names starting with the "Py" prefix. For example, before:
+
+```rust,ignore
+let err: PyErr = TypeError::py_err("error message");
+```
+
+After:
+
+```
+# use pyo3::{PyErr, PyResult, Python, type_object::PyTypeObject};
+# use pyo3::exceptions::{PyBaseException, PyTypeError};
+# Python::with_gil(|py| -> PyResult<()> {
+let err: PyErr = PyTypeError::new_err("error message");
+
+// Uses Display for PyErr, new for PyO3 0.12
+assert_eq!(err.to_string(), "TypeError: error message");
+
+// Now possible to interact with exception instances, new for PyO3 0.12
+let instance: &PyBaseException = err.instance(py);
+assert_eq!(instance.getattr("__class__")?, PyTypeError::type_object(py).as_ref());
+# Ok(())
+# }).unwrap();
+```
+
 ### `FromPy` has been removed
-To simplify the PyO3 public conversion trait hierarchy, the `FromPy` has been removed. In PyO3
-`0.11` there were two ways to define the to-Python conversion for a type: `FromPy<T> for PyObject`,
-and `IntoPy<PyObject> for T`.
+To simplify the PyO3 conversion traits, the `FromPy` trait has been removed. Previously there were
+two ways to define the to-Python conversion for a type:
+`FromPy<T> for PyObject` and `IntoPy<PyObject> for T`.
 
-Now, the canonical implementation is always `IntoPy`, so downstream crates may need to adjust
-accordingly.
+Now there is only one way to define the conversion, `IntoPy`, so downstream crates may need to
+adjust accordingly.
 
 Before:
 ```rust,ignore