Exclude mock archetypes/datatypes/components from release artifacts #2537
Closed
2 of 3 tasks
Labels
🌊 C++ API
C/C++ API specific
codegen/idl
🦟 regression
A thing that used to work in an earlier release
🎄 tracking issue
issue that tracks a bunch of subissues
Milestone
Comments
teh-cmc
added a commit
that referenced
this issue
Jun 30, 2023
… for combinatorial affixes (#2546) **Best reviewed on a commit-by-commit basis; in particular the `rerun codegen` commit is nothing but generated code.** This PR documents what we call "affixes", why they're problematic when contributing to the codegen stack and why we need to have regression tests in place for them. Quoting the newly added doc: > ### Understanding the subtleties of affixes > > So-called "affixes" are effects applied to objects defined with the Rerun IDL and that affect > the way these objects behave and interoperate with each other (so, yes, monads. shhh.). > > There are 3 distinct and very common affixes used when working with Rerun's IDL: transparency, > nullability and plurality. > > Broadly, we can describe these affixes as follows: > - Transparency allows for bypassing a single layer of typing (e.g. to "extract" a field out of > a struct). > - Nullability specifies whether a piece of data is allowed to be left unspecified at runtime. > - Plurality specifies whether a piece of data is actually a collection of that same type. > > We say "broadly" here because the way these affixes ultimately affect objects in practice will > actually depend on the kind of object that they are applied to, of which there are 3: archetypes, > components and datatypes. > > Not only that, but objects defined in Rerun's IDL are materialized into 3 distinct environments: > IDL definitions, Arrow datatypes and native code (e.g. Rust & Python). > > These environment have vastly different characteristics, quirks, pitfalls and limitations, > which once again lead to these affixes having different, sometimes surprising behavior > depending on the environment we're interested in. > Also keep in mind that Flatbuffers and native code are generally designed around arrays of > structures, while Arrow is all about structures of arrays! > > All in all, these interactions between affixes, object kinds and environments lead to a > combinatorial explosion of edge cases that can be very confusing when it comes to (de)serialization > code, and even API design. > > When in doubt, check out the `rerun.testing.archetypes.AffixFuzzer` IDL definitions, generated code and > test suites for definitive answers. At the moment, the test suite merely checks that the native objects that are generated make sense. Serialization roundtrips will be added in a couple PRs. It also only focuses on the Rust backend, as I'm still unsure what complex, deeply nested objects are gonna look like in the Python backend (need to work on #2390 first to get a feel for it). Also note that, for now, test-archetypes are mixed in with all other user-facing archetypes, which is not ideal. We'll work towards something better as the dust settle, as for now the overall organization of things is still a moving target (#2537). --- - #2484 - #2485 - #2487 - #2545 - #2546 - #2549 - #2554 - #2570 - #2571 --- <!-- This line will get updated when the PR build summary job finishes. --> PR Build Summary: https://build.rerun.io/pr/2487 <!-- pr-link-docs:start --> Docs preview: https://rerun.io/preview/46068dd/docs Examples preview: https://rerun.io/preview/46068dd/examples <!-- pr-link-docs:end -->
|
Becoming more and more urgent as the affix fuzzing tests grow exponentially with all the union stuff. |
teh-cmc
changed the title
12 tasks
|
Turned this into a tracking for all 3 supported languages. |
3 tasks
teh-cmc
added a commit
that referenced
this issue
Jul 25, 2023
Testing types now live under `re_types/src/testing`, which is itself feature gated behind the `testing` flag. Fixes #2809 Part of #2537 ### What ### Checklist * [x] I have read and agree to [Contributor Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and the [Code of Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md) * [x] I've included a screenshot or gif (if applicable) * [x] I have tested [demo.rerun.io](https://demo.rerun.io/pr/2810) (if applicable) - [PR Build Summary](https://build.rerun.io/pr/2810) - [Docs preview](https://rerun.io/preview/pr%3Acmc%2Frust_exclude_fuzzy/docs) - [Examples preview](https://rerun.io/preview/pr%3Acmc%2Frust_exclude_fuzzy/examples)
|
Can this be solved once and for all for all three languages by ignoring the test .fbs files in release builds? |
|
fbs files are only looked at at codegen time. Having different codegen for release & debug implies having two different libraries for debug and release, so that doesn't make sense. |
3 tasks
Wumpf
added a commit
that referenced
this issue
Aug 18, 2023
* Part of #2919 * Depends on #3001 * Solves C++ part of #2537 ### What Moves out all affix-fuzzer and co. testing types of the actual SDK. While working on that... * I figured out why warnings from arrow headers showed up only sometimes: When Arrow isn't added explicitely as a dependency, CMake doesn't seem to declare it as system header which normally suppresses warnings (as you can imagine this got really bad now when adding more cpps that include arrow to the test library). Obvious workaround is to add arrow explicitly to the text executable. * internal include paths got shorter/more natural in some cases. I had to make the `Includes` utility more clever in order to support types outside of the sdk, so we got this almost for free :) ### Checklist * [x] I have read and agree to [Contributor Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and the [Code of Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md) * [x] I've included a screenshot or gif (if applicable) * [x] I have tested [demo.rerun.io](https://demo.rerun.io/pr/3007) (if applicable) - [PR Build Summary](https://build.rerun.io/pr/3007) - [Docs preview](https://rerun.io/preview/pr%3Aandreas%2Fcpp%2Fseparate-test-types/docs) - [Examples preview](https://rerun.io/preview/pr%3Aandreas%2Fcpp%2Fseparate-test-types/examples)
3 tasks
teh-cmc
pushed a commit
that referenced
this issue
Sep 20, 2023
… log_deprecated (#3374) ### What Moves: - `rerun.log.error_utils` -> `rerun.error_utils` - `rerun.log.*` -> `rerun.log_deprecated.*` - `rerun.components.*` -> `rerun.components_deprecated.*` - `_rerun2.*` -> `rerun.*` Note, this required making the following changes in a few examples: * `rr.log.rects.RectFormat` -> `rr.RectFormat` * `rr.log.text.LoggingHandler` -> `rr.LoggingHandler` * `rr.log.annotation.AnnotationInfo` -> `rr.AnnotationInfo` Additionally, all of the AffixFuzzer is now generated into its own module in `rerun_py.tests.test_types` Does not yet move the new logging APIs out of `experimental`, but all of that can be managed in `rerun.__init__` now. * Resolves: #2537
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
We have a lot of objects that are only used for testing purposes and that generate a lot of useless code as far as end users are concerned. Make sure they don't pollute e.g. the wasm artifact!
UPDATE: Also so they don't pollute LSP output, autocomplete suggestions etc.
TODO:
The text was updated successfully, but these errors were encountered: