Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Python codegen: big cleaning and paving the way towards transforms #2603

Merged
merged 19 commits into from
Jul 5, 2023
Merged

Python codegen: big cleaning and paving the way towards transforms #2603

merged 19 commits into from
Jul 5, 2023

Conversation

abey79
Copy link
Member

@abey79 abey79 commented Jul 4, 2023 •
edited by github-actions bot
Loading

What

Note: as usual, review commit-by-commit (the bulk of the changes are in the first commit). This time, reviewers might want to spend some time on the auto-generated code, though, as this is a major focus of this PR.

This PR has the following aims:

  • Enforce and exploit the decision that a component must be a struct of one, and exactly one, field, which may be a datatype (in which case the component is known as "delegating", as its implementation delegates to the datatype) or a native type (e.g. float, int, etc.).
  • Significantly polish the Python SDK, with the goal of a cleaner, fully typed user-facing API, and a streamlined implementation. Part of this is achieved using attrs (see below).
  • Improve the from_similar() hook and introduce several others. The proposed mechanisms consist of storing the so-called (hand-coded) overrides in the _overrides sub-packages in each of the archetypes, components, and datatypes packages. These overrides are discovered by the code generator by "parsing" (well, "brute-force extracting symbols" from) _overrides/__init__.py [1]. The following hooks are currently implemented:
    • xxx_native_to_pa_array() (where xxx is the lower-cased name of the corresponding object): this is the primary way to flexibly convert native input into the Arrow array form of the component.
    • xxx_yyy_converter (yyy is a field name): optional converter function a given field of a native objects (e.g. Color). This enables the object to flexibly accept native input for the given field.
    • xxx_as_array: optional __array__() magic function for native objects (e.g. Point2D). This function enable the object to be interpreted as array data by Numpy, which can considerably simplify the implementation of, e.g., xxx_native_to_pa_array (Point2D is a good example).

This PR also introduces the following changes:

  • So far, the archetypes and log_any() were pulled into the rr namespace. This PR now pulls all component- and datatype-related symbols in rr.cmp and rr.dt, respectively. This is intended as a temporary solution, to avoid name clashes with the legacy API [2].
  • The attrs dependency is introduced as it allows much more flexibility than the standard library dataclass, in particular with the converter feature which I use extensively. Introducing new deps is always a tough call. In the case of attrs, it's most probably safe as this is one of the most common and well maintained package of the ecosystem (110+M dl/month).
  • Native objects automatically get __str__(), __int__(), or __float__() functions when they contain a single field of the related type.
  • The Point2D component now delegate to a new Point2D datatype, defined as a struct of two floats. This is done such as to avoid breaking compatibility with the current viewer. The long-term goal is to use the Vec2D datatype instead.
  • The CI now uses Python 3.11 for linting (and linting only, i.e. not testing, etc.)

Addressing union types is a non-goal of this PR. This will be part of the next PR (transforms). As such QuotedObject::from_union() is most likely broken.

TODO/TBD:

  • PyCharm's typechecker isn't happy with the Archetype, with is irritating. This could apparently be addressed by using xxxArrayLike instead of xxxArray annotations for archetype's fields. I'm curious how VSCode fares, though.
  • Maybe passing None as args to the archetype should set None instead of empty arrays (they aren't logged either way).
  • Maybe the notion of delegating vs. non-delegating component should be promoted to the langauge-agnostic code?

[1] I actually prototyped and tested an approach based on PyO3. It sorta worked, until I tried to import stuff (Numpy, ...) for the overrides. The codegen would somehow be given access to a venv with the required deps, etc. Hell no.
[2] Name clashes also exist internally to the new API, e.g. components.Point2D vs. datatypes.Point2D. This will have to be addressed before we can pull "everything" into the rr namespace.

Checklist

@abey79 @teh-cmc
Improved Python code generator
9c2bab1 
Co-authored-by: Clement Rey <cr.rey.clement@gmail.com>
@abey79 abey79 force-pushed the antoine/hope-python-cleanup-new branch from 22bd5a8 to b992ee7 Compare July 4, 2023 17:19
@abey79
Several project and typing fix
9d26e31 
- added missing attrs dependency
- CI: use Python 3.11 for python linting
- fixed a bunch of old-Python typing issue
@abey79
Added attrs to requirements-lint.txt
8719521
@abey79 abey79 requested a review from teh-cmc July 4, 2023 21:07
@abey79
Merge branch 'main' into antoine/hope-python-cleanup-new
87aa83c 
# Conflicts:
#	crates/re_types/source_hash.txt
#	crates/re_types_builder/src/codegen/python.rs
@abey79
source_hash.txt
6f986d3
@teh-cmc teh-cmc requested review from teh-cmc and removed request for teh-cmc July 5, 2023 09:08
@teh-cmc
Copy link
Member

teh-cmc commented Jul 5, 2023

Yeah my vim isn't too happy either... :(

image

teh-cmc
teh-cmc requested changes Jul 5, 2023
View reviewed changes
Copy link
Member

@teh-cmc teh-cmc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Niceeee! Need to bring those Fuzzy things back tho!

rerun_py/tests/unit/test_points2d.py Outdated Show resolved Hide resolved
rerun_py/tests/unit/test_points2d.py Show resolved Hide resolved
rerun_py/rerun_sdk/rerun/_rerun2/log_any.py Outdated Show resolved Hide resolved
crates/re_types/definitions/rerun/datatypes.fbs Outdated Show resolved Hide resolved
rerun_py/rerun_sdk/rerun/_rerun2/_baseclasses.py Outdated Show resolved Hide resolved
crates/re_types/definitions/rerun/testing/components/fuzzy.fbs Show resolved Hide resolved
crates/re_types_builder/src/codegen/python.rs Outdated Show resolved Hide resolved
crates/re_types_builder/src/codegen/python.rs Outdated Show resolved Hide resolved
crates/re_types/definitions/rerun/testing/components/fuzzy.fbs Outdated Show resolved Hide resolved
@abey79
Copy link
Member Author

abey79 commented Jul 5, 2023

Yeah my vim isn't too happy either... :(

image

The situation here is a bit weird typing-wise TBH (due to the args being extracted from the big arrays, etc.). How does it look in api_demo/main.py (whatever the new name is)?

@teh-cmc
Copy link
Member

teh-cmc commented Jul 5, 2023

Here's what I get for the roundtrip test:
image

Here's test_api:
image

@teh-cmc teh-cmc mentioned this pull request Jul 5, 2023
Merged
3 tasks
@teh-cmc teh-cmc self-requested a review July 5, 2023 16:32
teh-cmc
teh-cmc approved these changes Jul 5, 2023
View reviewed changes
rerun_py/rerun_sdk/rerun/_rerun2/_baseclasses.py
Comment on lines +100 to +127
"""
Converts native data into an Arrow array.

Subclasses must provide an implementation of this method (via an override) if they are to be used as either
an archetype's field (which should be the case for all components), or a (delegating) component's field (for
datatypes). Datatypes which are used only within other datatypes may omit implementing this method, provided
that the top-level datatype implements it.

A hand-coded override must be provided for the code generator to implement this method. The override must be
named `xxx_native_to_pa_array()`, where `xxx` is the lowercase name of the datatype. The override must be
located in the `_overrides` subpackage and *explicitly* imported by `_overrides/__init__.py` (to be noticed
by the code generator).

`color_native_to_pa_array()` in `_overrides/color.py` is a good example of how to implement this method, in
conjunction with the native type's converter (see `color_converter()`, used to construct the native `Color`
object).

Parameters
----------
data : T
The data to convert into an Arrow array.
data_type : pa.DataType
The Arrow data type of the data.

Returns
-------
The Arrow array encapsulating the data.
"""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice

@teh-cmc teh-cmc merged commit 59b20e9 into main Jul 5, 2023
@teh-cmc teh-cmc deleted the antoine/hope-python-cleanup-new branch July 5, 2023 17:14
teh-cmc added a commit that referenced this pull request Jul 5, 2023
@teh-cmc @abey79
Automatically assume arrow transparency for components (#2608)
f99bec5 
Fixes #2606 
On top of #2603 (requires single-field `Point2D` component)

### What

### Checklist
* [x] I have read and agree to [Contributor
Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and
the [Code of
Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md)
* [x] I've included a screenshot or gif (if applicable)
* [x] I have tested [demo.rerun.io](https://demo.rerun.io/pr/2608) (if
applicable)

- [PR Build Summary](https://build.rerun.io/pr/2608)
- [Docs
preview](https://rerun.io/preview/pr%3Acmc%2Fcomponent_always_arrow_transparent/docs)
- [Examples
preview](https://rerun.io/preview/pr%3Acmc%2Fcomponent_always_arrow_transparent/examples)

---------

Co-authored-by: Antoine Beyeler <antoine@rerun.io>
@jleibs jleibs mentioned this pull request Jul 27, 2023
Merged
3 tasks
jleibs added a commit that referenced this pull request Jul 27, 2023
@jleibs
Release 0.8 - Infrastructure investments and more transform improveme…
df05f49 
…nts - 2023-07-27 (#2842)

## [0.8.0](v0.7.0...v0.8.0) -
Infrastructure investments and more transform improvements - 2023-07-27

[Rerun](https://www.rerun.io/) is an easy-to-use visualization toolbox
for computer vision and robotics.

* Python: `pip install rerun-sdk`
* Rust: `cargo add rerun` and `cargo install rerun-cli`
* Online demo: <https://demo.rerun.io/version/0.8.0/>


### Overview & Highlights
- `log_pinhole` is now easier to use in simple cases and supports
non-RDF camera coordinates.
[#2614](#2614)
- You only need to set focal length and optional principal point instead
of setting the full 3x3 matrix.
- There is also a new argument: `camera_xyz` for setting the coordinate
system. The default is RDF (the old
default). This affects the visible camera frustum, how rays are
projected when hovering a 2D image, and how depth
   clouds are projected.
- The visualizer can now show coordinate arrows for all affine
transforms within the view.
[#2577](#2577)
- Linestrips and oriented bounding boxes can now be logged via batch
APIs in python.
- See: `log_linestrips_2d`, `log_linestrips_3d`,
[#2822](#2822) and `log_obbs`
[#2823](#2823)
- Rust users that build their own Viewer applications can now add fully
custom Space Views. Find more information
[here](https://www.rerun.io/docs/howto/extend-ui#custom-space-views-classes).
- New optional `flush_timeout` specifies how long Rerun will wait if a
TCP stream is disconnected during a flush.
[#2821](#2821)
- In Rust, `RecordingStream::connect` now requires `flush_timeout`
specified as an `Option<Duration>`.
- To keep default behavior, this can be specified using the
`rerun::default_flush_time()` helper.
- In Python `flush_init_sec` is now an optional argument to
`rr.connect()`
- In Rust, the `RecordingStream` now offers a stateful time API, similar
to the Python APIs. [#2506](#2506)
- You can now call `set_time_sequence`, `set_time_seconds`, and
`set_time_nanos` directly on the `RecordingStream`,
     which will set the time for all subsequent logs using that stream.
- This can be used as an alternative to the the previous
`MsgSender::with_time` APIs.
- The Rerun SDK now defaults to 8ms long microbatches instead of 50ms.
This makes the default behavior more suitable
for use-cases like real-time video feeds.
[#2220](#2220)
- Check out [the microbatching
docs](https://www.rerun.io/docs/reference/sdk-micro-batching) for more
information
   on fine-tuning the micro-batching behavior.
- The web viewer now incremental loads `.rrd` files when streaming over
HTTP. [#2412](#2412)

![Open Photogrammetry
Preview](https://static.rerun.io/9fa26e73a197690e0403cd35f29e31c2941dea36_release_080_photogrammetry_full.png)

### Ongoing Refactors
- There have been a number of significant internal changes going on
during this release with little visible impact.
This work will land across future releases, but is highlighted here
since much of it is visible through the
   changelog.
- The layout of the Viewer is now controlled by a Blueprint datastore.
In the future this will allow for direct API
control of the layout and configuration of the Viewer. A very early
prototype of this functionality is available
    via the `rerun.experimental` module in Python.
- An entirely new code-generation framework has been brought online for
Rust, Python and C++. This will eventually enable
new object-centric APIs with a more scalable, consistent, and ergonomic
experience.
- Bringup of C++ support is now underway and will eventually become our
third officially supported SDK language.

### Known Regressions
- Due to the Blueprint storage migration, blueprint persistence on web
is currently broken. Will be resolved in:
  [#2579](#2579)

### In Detail
#### 🐍 Python SDK
- Clean up warnings printed when `rr.init` hasn't been called
[#2209](#2209)
- Normalise Python typing syntax to 3.8+
[#2361](#2361)
- Simpler, sturdier stateful time tracking in both SDKs
[#2506](#2506)
- Fix not taking np.array for single colors
[#2569](#2569)
- Add a basic pyright config
[#2610](#2610)
- Improve `log_pinhole` and support non-RDF pinholes
[#2614](#2614)
- Expose batch APIs for linestrips
[#2822](#2822)
- Expose batch APIs for oriented bounding boxes
[#2823](#2823)

#### 🦀 Rust SDK
- Add example for adding custom Space Views
[#2328](#2328)
- Simpler, sturdier stateful time tracking in both SDKs
[#2506](#2506)
- Automagic flush when `take()`ing a `MemorySinkStorage`
[#2632](#2632)
- Logging SDK: Log warnings if user data is dropped
[#2630](#2630)
- Add support for `RecordingStream::serve`
[#2815](#2815)

#### 🌁 Viewer Improvements
- Better handle scroll-to-zoom in 3D views
[#1764](#1764)
- Add command to screenshot the application
[#2293](#2293)
- Show layout in blueprint tree view
[#2465](#2465)
- Double-click to select entity
[#2504](#2504)
- Add Rerun.io link/text in top bar
[#2540](#2540)
- New auto-layout of space views
[#2558](#2558)
- Add 'Dump datastore' command to palette
[#2564](#2564)
- Support any `dtype` for depth images
[#2602](#2602)
- Change "Save Selection" command to Cmd+Alt+S
[#2631](#2631)
- Consistent transform visualization for all entities with transforms
[#2577](#2577)
- Improve `log_pinhole` and support non-RDF pinholes
[#2614](#2614)

#### 🚀 Performance Improvements
- Flush the batches every 8ms instead of 50 ms
[#2220](#2220)
- Replace `image` crate jpeg decoder with zune-jpeg
[#2376](#2376)
- Stream `.rrd` files when loading via http
[#2412](#2412)

#### 🪳 Bug Fixes
- Fix deadlock when misusing the Caches
[#2318](#2318)
- Fix unstable order/flickering of "shown in" space view list on
selection [#2327](#2327)
- Fix transforms not applied to connections from transform context
[#2407](#2407)
- Fix texture clamping and color gradient selection being displayed
incorrectly [#2394](#2394)
- Fix projected ray length
[#2482](#2482)
- Tweak the depth bias multiplier for WebGL
[#2491](#2491)
- Clip image zoom rectangle
[#2505](#2505)
- Fix missing feature flags for benchmarks
[#2515](#2515)
- `run_all.py` script fixes
[#2519](#2519)
- Update egui_tiles with fix for drag-and-drop-panic
[#2555](#2555)
- Convert objectron proto.py back to using typing.List
[#2559](#2559)
- Exclude from `objectron/proto/objectron/proto.py` from `just
py-format` [#2562](#2562)
- Fix pinhole visualization not working with camera extrinsics &
intrinsics on the same path
[#2568](#2568)
- Fix: always auto-layout spaceviews until the user interveens
[#2583](#2583)
- Fix freeze/crash when logging large times
[#2588](#2588)
- Update egui_tiles to fix crash
[#2598](#2598)
- Fix clicking object with single instance (of every component)
selecting instance instead of entity
[#2573](#2573)
- Cleanup internal data-structures when process has been forked
[#2676](#2676)
- Fix shutdown race-condition by introducing a flush_timeout before
dropping data [#2821](#2821)
- Fix ui-scale based point/line sizes incorrectly scaled when zooming
based on horizontal dimension
[#2805](#2805)
- Fix visibility toggle for maximized Space Views
[#2806](#2806)
- Fix loading file via CLI
[#2807](#2807)
- Fix disconnected space APIs in Python SDK
[#2832](#2832)
- Avoid unwrap when generating authkey
[#2804](#2804)

#### 🧑‍🏫 Examples
- Add example template
[#2392](#2392)
- Show hidden url search param in `app.rerun.io`
[#2455](#2455)
- Minimal example of running an intel realsense depth sensor live
[#2541](#2541)
- Add a simple example to display Open Photogrammetry Format datasets
[#2512](#2512)
- Move `examples/api_demo` -> `tests/test_api`
[#2585](#2585)

#### 📚 Docs
- Docs: link to `rr.save` and suggest `rerun` instead of `python -m
rerun` [#2586](#2586)
- Update docs about transforms
[#2496](#2496)
- Fixup remaining usages of log_rigid3 in docs
[#2831](#2831)

#### 🎨 Renderer Improvements
- Expose type erased draw data that can be consumed directly
[#2300](#2300)
- Use less `mut` when using `RenderContext`
[#2312](#2312)

#### 🧑‍💻 Dev-experience
- Better error messages in build.rs
[#2173](#2173)
- Recommend sccache in CONTRIBUTING.md
[#2245](#2245)
- introduce `re_tracing`
[#2283](#2283)
- lint: standardize formatting of let-else-return statements
[#2297](#2297)
- Centralized build tools in `re_build_tools`
[#2331](#2331)
- Lint for explicit quotes
[#2332](#2332)
- Added example screenshot instructions in `just upload --help`
[#2454](#2454)
- Added support for puling image from an URL to `upload_image.py`
[#2462](#2462)
- `setup_dev.sh` now installs pngcrush
[#2470](#2470)
- Added docs/code-examples to the directories checked by py-lint and
py-format [#2476](#2476)
- Link to demo in PR + check checkboxes
[#2543](#2543)
- Add script to find external issues we haven't commented on
[#2532](#2532)
- Move CI-related scripts to its own folder
[#2561](#2561)
- Render PR description as template
[#2563](#2563)
- Add basic testing automation against all version of Python using nox
[#2536](#2536)
- Run clippy on public API too
[#2596](#2596)
- Bump all `py-lint`-related package versions
[#2600](#2600)
- Crates publishing script
[#2604](#2604)
- Fix rust docs deploy
[#2615](#2615)
- Add support for .gitignore to scripts/lint.py
[#2666](#2666)

#### 🗣 Refactors
- Refactor space-view dependencies:
- Move spatial space view to its own crate
[#2286](#2286)
- Separate crate for bar chart space view
[#2322](#2322)
- Separate crate for time series space view
[#2324](#2324)
- Separate crate for tensor space view
[#2334](#2334)
- Separate viewport related files out to a new re_viewport crate
[#2251](#2251)
- Remove timepanel dependency from viewport
[#2256](#2256)
- New trait system for SpaceViews:
- Initial Space View trait & port of text space views to the new Space
View trait system [#2281](#2281)
- Extend/iterate on SpaceViewClass framework with SceneContext & port
SpatialSpaceView scene parts
[#2304](#2304)
- Finalize move of SpatialSpaceView to SpaceViewClass trait framework
[#2311](#2311)
- Typename cleanup in SpaceViewClass framework
[#2321](#2321)
- Automatic fallback for unrecognized Space View Class, start removing
old ViewCategory [#2357](#2357)
- Rename ScenePart -> ViewPartSystem + related renamings
[#2674](#2674)
- Dynamically registered space view (part/context) systems
[#2688](#2688)
- Viewer's command queue is now a channel, allowing to queue commands
without mutable access
[#2339](#2339)
- Break up app.rs into parts
[#2303](#2303)
- Break out `re_log_types::component_types` as `re_components`
[#2258](#2258)
- Introduce StoreHub and rename Recording->Store
[#2301](#2301)
- Move StoreHub out of the Viewer during Update
[#2330](#2330)
- Expand CommandSender to support SystemCommand
[#2344](#2344)
- Use `camino` crate for UTF8 paths in `re_types_builder`
[#2637](#2637)
- Separate 2d & 3d spaceview classes, removal of `ViewCategory`,
`SpaceViewClass` driven spawn heuristics
[#2716](#2716)
- Move object property heuristics to heuristics.rs
[#2764](#2764)

#### 📦 Dependencies
- Version `rand` & friends at workspace level
[#2508](#2508)
- Update to PyO3 0.19
[#2350](#2350)
- Pin `half` to `2.2.1`
[#2587](#2587)

#### 📘 Blueprint Changes
- Drive blueprints off of a DataStore
[#2010](#2010)
- Split SpaceView -> SpaceViewState + SpaceViewBlueprint
[#2188](#2188)
- Split the Blueprint into AppBlueprint and ViewportBlueprint
[#2358](#2358)
- Swap the naming of Viewport and ViewportBlueprint
[#2595](#2595)
- Basic persistence for blueprints
[#2578](#2578)

#### 🏭 New Codegen Framework
- Codegen/IDL 1: add more build tools
[#2362](#2362)
- Codegen/IDL 2: introduce `re_types_builder`
[#2363](#2363)
- Codegen/IDL 3: introduce `re_types`
[#2369](#2369)
- Codegen/IDL 4: definitions for a `Points2D` archetype
[#2370](#2370)
- Codegen/IDL 5: auto-generated Python code for `Points2D`
[#2374](#2374)
- Codegen/IDL 7: handwritten Python tests and extensions for `Points2D`
[#2410](#2410)
- Codegen/IDL 6: auto-generated Rust code for `Points2D`
[#2375](#2375)
- Codegen/IDL 8: handwritten Rust tests and extensions for `Points2D`
[#2432](#2432)
- Codegen'd Rust/Arrow 1: upgrading to actual `TokenStream`s
[#2484](#2484)
- Codegen'd Rust/Arrow 2: matching legacy definitions
[#2485](#2485)
- Codegen'd Rust/Arrow 3: misc fixes & improvements
[#2487](#2487)
- Codegen'd Rust/Arrow 4: out-of-sync definitions CI detection
[#2545](#2545)
- Codegen'd Rust/Arrow 5: doc, definitions and regression tests for
combinatorial affixes
[#2546](#2546)
- Codegen'd Rust/Arrow 6: serialization
[#2549](#2549)
- Codegen'd Rust/Arrow 7: deserialization
[#2554](#2554)
- Codegen'd Rust/Arrow 8: carry extension metadata across transparency
layers [#2570](#2570)
- Codegen'd Rust/Arrow 9: Rust backport!
[#2571](#2571)
- End-to-end cross-language roundtrip tests for our archetypes
[#2601](#2601)
- Automatically derive `Debug` and `Clone` in Rust backend
[#2613](#2613)
- Generating (de)serialization code for dense unions in Rust backend
[#2626](#2626)
- Fix `FixedSizeList` deserialization edge-case + trivial optimizations
[#2673](#2673)
- Make `Datatype` & `Component` both inherit from `Loggable`
[#2677](#2677)
- Roundtrip-able `Transform3D`s
[#2669](#2669)
- Don't inline recursive datatypes in Rust backend
[#2760](#2760)
- Automatically derive `tuple_struct` attr and trivial `From` impls
where possible [#2772](#2772)
- Introduce roundtrip-able `Points3D` archetype (py + rs)
[#2774](#2774)
- Add `fmt::Debug` implementations to various types.
[#2784](#2784) (thanks
[@kpreid](https://github.com/kpreid)!)
- Isolate testing types in Rust backend
[#2810](#2810)
- Fix out-of-sync codegen hash
[#2567](#2567)
- Python backport: add `log_any()`
[#2581](#2581)
- Integrate unit examples into codegen stack
[#2590](#2590)
- Disable codegen on windows
[#2592](#2592)
- Python codegen: big cleaning and paving the way towards transforms
[#2603](#2603)
- Automatically assume arrow transparency for components
[#2608](#2608)
- Fix wrong path being `rerun_if_changed()` in `compute_dir_hash`
[#2612](#2612)
- Support transparency at the semantic layer
[#2611](#2611)
- Don't use builtin `required` anymore, introduce `nullable` instead
[#2619](#2619)
- Rust codegen: generate proper docstrings
[#2668](#2668)
- Support nullable Arrow unions using virtual union arms
[#2708](#2708)
- Introduce support for querying Archetypes
[#2743](#2743)
- Introduce legacy shims and migrate DataCell to re_types::Component
[#2752](#2752)

#### 🌊 Starting work on C++
- Seed of C and C++ SDKs
[#2594](#2594)
- Move C++ SDK to own folder
[#2624](#2624)
- C++ codegen [#2678](#2678)
- C++ codegen for reporting arrow data type for structs
[#2756](#2756)
- Don't inline recursive datatypes in C++ backend
[#2765](#2765)
- C++ codegen to_arrow_data_type for unions
[#2766](#2766)
- C++ codegen arrow serialize non-union components/datatypes without
nested rerun types [#2820](#2820)
- C++ codegen of structs and unions
[#2707](#2707)
- Fix cpp formatter differences
[#2773](#2773)

#### 🤷‍♂️ Other
- test_api: set different app_id based on what test is run
[#2599](#2599)
- Introduce `rerun compare` to check whether 2 rrd files are
functionally equivalent
[#2597](#2597)
- Remove `files.exclude` in vscode settings
[#2621](#2621)
- Support feature-gated rust attributes
[#2813](#2813)

### Checklist
* [x] I have read and agree to [Contributor
Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and
the [Code of
Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md)
* [x] I've included a screenshot or gif (if applicable)
* [x] I have tested [demo.rerun.io](https://demo.rerun.io/pr/2842) (if
applicable)

- [PR Build Summary](https://build.rerun.io/pr/2842)
- [Docs preview](https://rerun.io/preview/pr%3Arelease-0.8/docs)
- [Examples preview](https://rerun.io/preview/pr%3Arelease-0.8/examples)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@emilk emilk emilk left review comments

@teh-cmc teh-cmc teh-cmc approved these changes

Assignees
No one assigned
Labels
codegen/idl
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@abey79 @teh-cmc @emilk