Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Unhandled promise rejection: undefined thrown in CI #12267

Merged
merged 2 commits into from
Oct 30, 2024

Conversation

ggetz
Copy link
Contributor

@ggetz ggetz commented Oct 29, 2024

Description

I think I caught the race condition: Basically whenever the camera moved or we started rendering a tileset, there were tiles that started load. When the test ended, the tileset was destroyed, and the error was thrown. So we'll need to await the tiles loaded helper to ensure these test pass.

Issue number and link

Fixes #11958

Testing plan

CI should pass, even if you re-run!

Author checklist

  • I have submitted a Contributor License Agreement
  • I have added my name to CONTRIBUTORS.md
  • I have updated CHANGES.md with a short summary of my change
  • I have added or updated unit tests to ensure consistent code coverage
  • I have updated the inline documentation, and included code examples where relevant
  • I have performed a self-review of my code

Copy link

Thank you for the pull request, @ggetz!

✅ We can confirm we have a CLA on file for you.

@lukemckinstry
Copy link
Contributor

I ran the tests twice and got the same 8 test failures each time. Are these related? I do not remember seeing these before.

1) creates environment map and spherical harmonics at surface in Philadelphia with static lighting
     Scene/DynamicEnvironmentMapManager render tests
     Expected (0.11088263988494873, 0.13940687477588654, 0.17248643934726715) to equal epsilon (0.11017649620771408, 0.13869766891002655, 0.17165547609329224), 0.0001.
    at <Jasmine>
    at UserContext.<anonymous> (packages/engine/Specs/Scene/DynamicEnvironmentMapManagerSpec.js:94:58 <- Build/Specs/SpecList.js:158512:58)
    at <Jasmine>
     Expected (0.08825122565031052, 0.11136562377214432, 0.15215623378753662) to equal epsilon (0.08705271780490875, 0.11016352474689484, 0.15077166259288788), 0.0001.
    at <Jasmine>
    at UserContext.<anonymous> (packages/engine/Specs/Scene/DynamicEnvironmentMapManagerSpec.js:102:58 <- Build/Specs/SpecList.js:158520:58)
    at <Jasmine>

2) lighting uses atmosphereScatteringIntensity value
     Scene/DynamicEnvironmentMapManager render tests
     Expected (0.03268122673034668, 0.03987933322787285, 0.0481969378888607) to equal epsilon (0.0322723351418972, 0.039464931935071945, 0.047749463468790054), 0.0001.
    at <Jasmine>
    at UserContext.<anonymous> (packages/engine/Specs/Scene/DynamicEnvironmentMapManagerSpec.js:602:58 <- Build/Specs/SpecList.js:158918:58)
    at <Jasmine>
     Expected (0.026692016050219536, 0.03257599472999573, 0.04299120232462883) to equal epsilon (0.025989927351474762, 0.031872138381004333, 0.04223670810461044), 0.0001.
    at <Jasmine>
    at UserContext.<anonymous> (packages/engine/Specs/Scene/DynamicEnvironmentMapManagerSpec.js:610:58 <- Build/Specs/SpecList.js:158926:58)
    at <Jasmine>

3) lighting uses gamma value
     Scene/DynamicEnvironmentMapManager render tests
     Expected (0.18760140240192413, 0.21416431665420532, 0.2373003512620926) to equal epsilon (0.18712928891181946, 0.21367456018924713, 0.23666927218437195), 0.0001.
    at <Jasmine>
    at UserContext.<anonymous> (packages/engine/Specs/Scene/DynamicEnvironmentMapManagerSpec.js:684:58 <- Build/Specs/SpecList.js:158984:58)
    at <Jasmine>
     Expected (0.13648082315921783, 0.15867212414741516, 0.19189152121543884) to equal epsilon (0.13568174839019775, 0.15787045657634735, 0.19085952639579773), 0.0001.
    at <Jasmine>
    at UserContext.<anonymous> (packages/engine/Specs/Scene/DynamicEnvironmentMapManagerSpec.js:692:58 <- Build/Specs/SpecList.js:158992:58)
    at <Jasmine>

4) lighting uses brightness value
     Scene/DynamicEnvironmentMapManager render tests
     Expected (0.0606422945857048, 0.07502496987581253, 0.09167612344026566) to equal epsilon (0.05981340631842613, 0.07419705390930176, 0.09077795594930649), 0.0001.
    at <Jasmine>
    at UserContext.<anonymous> (packages/engine/Specs/Scene/DynamicEnvironmentMapManagerSpec.js:766:58 <- Build/Specs/SpecList.js:159050:58)
    at <Jasmine>
     Expected (0.05300987884402275, 0.06477546691894531, 0.08560837805271149) to equal epsilon (0.051604993641376495, 0.06336799263954163, 0.08409948647022247), 0.0001.
    at <Jasmine>
    at UserContext.<anonymous> (packages/engine/Specs/Scene/DynamicEnvironmentMapManagerSpec.js:774:58 <- Build/Specs/SpecList.js:159058:58)
    at <Jasmine>

5) lighting uses saturation value
     Scene/DynamicEnvironmentMapManager render tests
     Expected (0.1357308030128479, 0.1357308030128479, 0.1357308030128479) to equal epsilon (0.13499368727207184, 0.13499368727207184, 0.13499368727207184), 0.0001.
    at <Jasmine>
    at UserContext.<anonymous> (packages/engine/Specs/Scene/DynamicEnvironmentMapManagerSpec.js:848:58 <- Build/Specs/SpecList.js:159116:58)
    at <Jasmine>
     Expected (0.10941863805055618, 0.10941863805055618, 0.10941863805055618) to equal epsilon (0.1081928238272667, 0.1081928238272667, 0.1081928238272667), 0.0001.
    at <Jasmine>
    at UserContext.<anonymous> (packages/engine/Specs/Scene/DynamicEnvironmentMapManagerSpec.js:856:58 <- Build/Specs/SpecList.js:159124:58)
    at <Jasmine>

6) lighting uses ground color value
     Scene/DynamicEnvironmentMapManager render tests
     Expected (0.13458813726902008, 0.12055020779371262, 0.16090303659439087) to equal epsilon (0.1342056840658188, 0.11958353966474533, 0.15991388261318207), 0.0001.
    at <Jasmine>
    at UserContext.<anonymous> (packages/engine/Specs/Scene/DynamicEnvironmentMapManagerSpec.js:930:58 <- Build/Specs/SpecList.js:159182:58)
    at <Jasmine>
     Expected (0.07639303803443909, 0.12079824507236481, 0.15795058012008667) to equal epsilon (0.07575193047523499, 0.11915278434753418, 0.15629366040229797), 0.0001.
    at <Jasmine>
    at UserContext.<anonymous> (packages/engine/Specs/Scene/DynamicEnvironmentMapManagerSpec.js:938:58 <- Build/Specs/SpecList.js:159190:58)
    at <Jasmine>
     Expected 0.00002864763700927142 to be less than 0.
    at <Jasmine>
    at UserContext.<anonymous> (packages/engine/Specs/Scene/DynamicEnvironmentMapManagerSpec.js:965:60 <- Build/Specs/SpecList.js:159216:60)
    at <Jasmine>
     Expected 0.00011105152952950448 to be less than 0.
    at <Jasmine>
    at UserContext.<anonymous> (packages/engine/Specs/Scene/DynamicEnvironmentMapManagerSpec.js:968:60 <- Build/Specs/SpecList.js:159218:60)
    at <Jasmine>
     Expected 0.0001212774368468672 to be less than 0.
    at <Jasmine>
    at UserContext.<anonymous> (packages/engine/Specs/Scene/DynamicEnvironmentMapManagerSpec.js:969:60 <- Build/Specs/SpecList.js:159219:60)
    at <Jasmine>

7) lighting uses ground albedo value
     Scene/DynamicEnvironmentMapManager render tests
     Expected (0.15290607511997223, 0.18143033981323242, 0.19807767868041992) to equal epsilon (0.15277373790740967, 0.1812949925661087, 0.19759616255760193), 0.0001.
    at <Jasmine>
    at UserContext.<anonymous> (packages/engine/Specs/Scene/DynamicEnvironmentMapManagerSpec.js:1012:58 <- Build/Specs/SpecList.js:159248:58)
    at <Jasmine>
     Expected (0.06722989678382874, 0.09034423530101776, 0.13935478031635284) to equal epsilon (0.0670194923877716, 0.09013032913208008, 0.13857196271419525), 0.0001.
    at <Jasmine>
    at UserContext.<anonymous> (packages/engine/Specs/Scene/DynamicEnvironmentMapManagerSpec.js:1020:58 <- Build/Specs/SpecList.js:159256:58)
    at <Jasmine>

8) picks based on batchId
     Scene/Model/Model3DTileContent pnts
     Expected not to render [0,0,0,255], but actually rendered [0,0,0,255].
    at <Jasmine>
    at packages/engine/Specs/Scene/Model/Model3DTileContentSpec.js:982:27 <- Build/Specs/SpecList.js:256492:28
    at compare (Specs/addDefaultMatchers.js:387:13 <- Build/Specs/karma-main.js:355:13)
    at <Jasmine>

@ggetz
Copy link
Contributor Author

ggetz commented Oct 29, 2024

I ran the tests twice and got the same 8 test failures each time. Are these related? I do not remember seeing these before.

These are unrelated. The lighting specs were added in #12129, and I can loosen the epsilon checks. I'll open a new PR and tag you for review.

Copy link
Contributor

@lukemckinstry lukemckinstry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no longer seeing the errors 👍

@lukemckinstry lukemckinstry merged commit ea6e34f into main Oct 30, 2024
10 checks passed
@lukemckinstry lukemckinstry deleted the fix-3d-tileset-failures branch October 30, 2024 14:59
@javagl
Copy link
Contributor

javagl commented Oct 30, 2024

Basically all the tests are using the pattern

it("works", function () {
  // Returns pending exception
  return Cesium3DTilesTester.loadTileset(scene, tilesetEmptyRootUrl).then(
    function (tileset) {
      // Expectations here
    }
  });
})

This was changed for one case now (namely, the one before the one that was xit'ed out), at https://github.com/CesiumGS/cesium/pull/12267/files#diff-d583bed57d63fe55e0948b41083f208cf988cca081e41845659480a3ccbfefc8R675

  1. Why? 😆
  2. More importantly: Could problems (like race conditions) in general be caused by the fact that the tests are all creating a promise that is not waited for?

I wonder whether it could make sense to systematically and mechanically chage the pattern from above into

it("works", async function() {
  const tileset = await Cesium3DTilesTester.loadTileset(scene, tilesetEmptyRootUrl);
  // Expectations here
  // (No pending promise returned)
})

?
(Subjectively, I'd say that this could be part of "modernization", "cleanup", and would just be "nicer" (with less indentation and increased readability), but maybe there is a profound technical reason as well)

@ggetz
Copy link
Contributor Author

ggetz commented Nov 5, 2024

This was changed for one case now (namely, the one before the one that was xit'ed out), at

I assume you are talking about the change from a promise chain to async/await?

This was changed because async/await is more readable than nested promise chains, and we'd like to opt for it where we can. Since the codebase has MANY instances of promise chains throughout since we were using promises far before async/await syntax was available for use, I'd be hesitant to add linting for it right now, but eventually we might want to add a rule like prefer-await-to-then.

I don't think this changes the behavior of the test or prevents race conditions, other than just making things more readable and hopefully less error prone. Async functions implicitly always return a promise.

@javagl
Copy link
Contributor

javagl commented Nov 5, 2024

I don't think this changes the behavior of the test or prevents race conditions, other than just making things more readable and hopefully less error prone. Async functions implicitly always return a promise.

This refers to some guesses about where race conditions come from. And that's difficult. (I knew that since I wrote my first multi-threaded program that just printed "Helol Wordl!" 🤡 ). Specifically, it crucially depends on how Jasmine is working internally, and I don't have a clue about that.

But the thought was revolving about a pattern like this:

// Some object that is SHARED (!) between tests
let scene = ...;

it("runA", function () {
  // This may depend on or modify(!) the state of the shared scene
  return Cesium3DTilesTester.loadTileset(scene, ... ).then( ... );
})
it("runB", function () {
  // This may depend on or modify(!) the state of the shared scene
  return Cesium3DTilesTester.loadTileset(scene, ... ).then( ... );
})

Now, how does Jasmine execute this?

The tests are not async - so for Jasmine, they look like "fire and forget" tests. And I could imaine that it starts runA, and then starts runB before the then-part of runA is executed - leading to a race condition from the concurrent access to the shared scene object.

If both the tests contained the await for the tileset, then they would be async. And in this case, Jasmine would have to await them to be finished, which might prevent the race condition. (It would make the tests "atomic", so to speak, from a perspective of their execution...)

Wild guesses, sure but ... it's not totally unreasonable, is it?

@jjspace
Copy link
Contributor

jjspace commented Nov 5, 2024

Now, how does Jasmine execute this?
The tests are not async

If a spec returns a Promise, like loadTileset().then(), then Jasmine treats it as an async test and (as I understand it) will wait for it to finish before it moves on https://jasmine.github.io/tutorials/async

They also have a bunch of async related answers in the FAQ https://jasmine.github.io/pages/faq.html#async

If both the tests contained the await for the tileset, then they would be async

As Gabby stated:

Async functions implicitly always return a promise.

As long as they return a promise they should be treated the same way as writing it with async/await

@javagl
Copy link
Contributor

javagl commented Nov 5, 2024

OK, from quickly looking over this, the part at https://jasmine.github.io/pages/faq.html#late-failures , seems to ...

  1. describe exactly what I drafted in the above example
  2. describe exactly what we saw here

And about the statement from the tutorial page:

If you need more control, you can explicitly return a promise instead. Jasmine considers any object with a then method to be a promise

That return is obviously important.

I think that this is on the radar, and it may be unlikely to be a widespread flaw, but ... one instance of this flaw would be enough. So one review pass for the current specs could be to carefully go though each of them, and see whether there is any case where the code is

    it("verify memory usage statistics", function () {
      // 10 buildings, 36 ushort indices and 24 vertices per building, 6 float
      // components (position, normal) and 1 uint component (batchId) per vertex
      // 10 * [(24 * (6 * 4 + 1 * 4)) + (36 * 2)] = 7440 bytes
      const singleTileGeometryMemory = 7440;
      const singleTileTextureMemory = 0;
      const singleTileBatchTextureMemory = 40;
      const singleTilePickTextureMemory = 40;
      const tilesLength = 5;

      viewNothing();

      Cesium3DTilesTester.loadTileset(scene, tilesetUrl).then(
        function (tileset) {
          const statistics = tileset._statistics;

          // No tiles loaded
          expect(statistics.geometryByteLength).toEqual(0);
          expect(statistics.texturesByteLength).toEqual(0);
          expect(statistics.batchTableByteLength).toEqual(0);

          viewRootOnly();
          return Cesium3DTilesTester.waitForTilesLoaded(scene, tileset).then(
            function () {
              // Root tile loaded
              expect(statistics.geometryByteLength).toEqual(
                singleTileGeometryMemory,
              );
              expect(statistics.texturesByteLength).toEqual(
                singleTileTextureMemory,
              );
              expect(statistics.batchTableByteLength).toEqual(0);

              viewAllTiles();
              return Cesium3DTilesTester.waitForTilesLoaded(
                scene,
                tileset,
              ).then(function () {
                // All tiles loaded
                expect(statistics.geometryByteLength).toEqual(
                  singleTileGeometryMemory * tilesLength,
                );
                expect(statistics.texturesByteLength).toEqual(
                  singleTileTextureMemory * tilesLength,
                );
                expect(statistics.batchTableByteLength).toEqual(0);

                // One feature colored, the batch table memory is now higher
                tileset.root.content.getFeature(0).color = Color.RED;
                scene.renderForSpecs();
                expect(statistics.geometryByteLength).toEqual(
                  singleTileGeometryMemory * tilesLength,
                );
                expect(statistics.texturesByteLength).toEqual(
                  singleTileTextureMemory * tilesLength,
                );
                expect(statistics.batchTableByteLength).toEqual(
                  singleTileBatchTextureMemory,
                );

                // All tiles picked, the texture memory is now higher
                scene.pickForSpecs();
                expect(statistics.geometryByteLength).toEqual(
                  singleTileGeometryMemory * tilesLength,
                );
                expect(statistics.texturesByteLength).toEqual(
                  singleTileTextureMemory * tilesLength,
                );
                expect(statistics.batchTableByteLength).toEqual(
                  singleTileBatchTextureMemory +
                    singleTilePickTextureMemory * tilesLength,
                );

                // Tiles are still in memory when zoomed out
                viewNothing();
                scene.renderForSpecs();
                expect(statistics.geometryByteLength).toEqual(
                  singleTileGeometryMemory * tilesLength,
                );
                expect(statistics.texturesByteLength).toEqual(
                  singleTileTextureMemory * tilesLength,
                );
                expect(statistics.batchTableByteLength).toEqual(
                  singleTileBatchTextureMemory +
                    singleTilePickTextureMemory * tilesLength,
                );

                // Trim loaded tiles, expect the memory statistics to be 0
                tileset.trimLoadedTiles();
                scene.renderForSpecs();
                expect(statistics.geometryByteLength).toEqual(0);
                expect(statistics.texturesByteLength).toEqual(0);
                expect(statistics.batchTableByteLength).toEqual(0);
              });
            },
          );
        },
      );
    });

or

    it("verify memory usage statistics", function () {
      // 10 buildings, 36 ushort indices and 24 vertices per building, 6 float
      // components (position, normal) and 1 uint component (batchId) per vertex
      // 10 * [(24 * (6 * 4 + 1 * 4)) + (36 * 2)] = 7440 bytes
      const singleTileGeometryMemory = 7440;
      const singleTileTextureMemory = 0;
      const singleTileBatchTextureMemory = 40;
      const singleTilePickTextureMemory = 40;
      const tilesLength = 5;

      viewNothing();

      return Cesium3DTilesTester.loadTileset(scene, tilesetUrl).then(
        function (tileset) {
          const statistics = tileset._statistics;

          // No tiles loaded
          expect(statistics.geometryByteLength).toEqual(0);
          expect(statistics.texturesByteLength).toEqual(0);
          expect(statistics.batchTableByteLength).toEqual(0);

          viewRootOnly();
          return Cesium3DTilesTester.waitForTilesLoaded(scene, tileset).then(
            function () {
              // Root tile loaded
              expect(statistics.geometryByteLength).toEqual(
                singleTileGeometryMemory,
              );
              expect(statistics.texturesByteLength).toEqual(
                singleTileTextureMemory,
              );
              expect(statistics.batchTableByteLength).toEqual(0);

              viewAllTiles();
              Cesium3DTilesTester.waitForTilesLoaded(
                scene,
                tileset,
              ).then(function () {
                // All tiles loaded
                expect(statistics.geometryByteLength).toEqual(
                  singleTileGeometryMemory * tilesLength,
                );
                expect(statistics.texturesByteLength).toEqual(
                  singleTileTextureMemory * tilesLength,
                );
                expect(statistics.batchTableByteLength).toEqual(0);

                // One feature colored, the batch table memory is now higher
                tileset.root.content.getFeature(0).color = Color.RED;
                scene.renderForSpecs();
                expect(statistics.geometryByteLength).toEqual(
                  singleTileGeometryMemory * tilesLength,
                );
                expect(statistics.texturesByteLength).toEqual(
                  singleTileTextureMemory * tilesLength,
                );
                expect(statistics.batchTableByteLength).toEqual(
                  singleTileBatchTextureMemory,
                );

                // All tiles picked, the texture memory is now higher
                scene.pickForSpecs();
                expect(statistics.geometryByteLength).toEqual(
                  singleTileGeometryMemory * tilesLength,
                );
                expect(statistics.texturesByteLength).toEqual(
                  singleTileTextureMemory * tilesLength,
                );
                expect(statistics.batchTableByteLength).toEqual(
                  singleTileBatchTextureMemory +
                    singleTilePickTextureMemory * tilesLength,
                );

                // Tiles are still in memory when zoomed out
                viewNothing();
                scene.renderForSpecs();
                expect(statistics.geometryByteLength).toEqual(
                  singleTileGeometryMemory * tilesLength,
                );
                expect(statistics.texturesByteLength).toEqual(
                  singleTileTextureMemory * tilesLength,
                );
                expect(statistics.batchTableByteLength).toEqual(
                  singleTileBatchTextureMemory +
                    singleTilePickTextureMemory * tilesLength,
                );

                // Trim loaded tiles, expect the memory statistics to be 0
                tileset.trimLoadedTiles();
                scene.renderForSpecs();
                expect(statistics.geometryByteLength).toEqual(0);
                expect(statistics.texturesByteLength).toEqual(0);
                expect(statistics.batchTableByteLength).toEqual(0);
              });
            },
          );
        },
      );
    });

instead of

    it("verify memory usage statistics", function () {
      // 10 buildings, 36 ushort indices and 24 vertices per building, 6 float
      // components (position, normal) and 1 uint component (batchId) per vertex
      // 10 * [(24 * (6 * 4 + 1 * 4)) + (36 * 2)] = 7440 bytes
      const singleTileGeometryMemory = 7440;
      const singleTileTextureMemory = 0;
      const singleTileBatchTextureMemory = 40;
      const singleTilePickTextureMemory = 40;
      const tilesLength = 5;

      viewNothing();

      return Cesium3DTilesTester.loadTileset(scene, tilesetUrl).then(
        function (tileset) {
          const statistics = tileset._statistics;

          // No tiles loaded
          expect(statistics.geometryByteLength).toEqual(0);
          expect(statistics.texturesByteLength).toEqual(0);
          expect(statistics.batchTableByteLength).toEqual(0);

          viewRootOnly();
          return Cesium3DTilesTester.waitForTilesLoaded(scene, tileset).then(
            function () {
              // Root tile loaded
              expect(statistics.geometryByteLength).toEqual(
                singleTileGeometryMemory,
              );
              expect(statistics.texturesByteLength).toEqual(
                singleTileTextureMemory,
              );
              expect(statistics.batchTableByteLength).toEqual(0);

              viewAllTiles();
              return Cesium3DTilesTester.waitForTilesLoaded(
                scene,
                tileset,
              ).then(function () {
                // All tiles loaded
                expect(statistics.geometryByteLength).toEqual(
                  singleTileGeometryMemory * tilesLength,
                );
                expect(statistics.texturesByteLength).toEqual(
                  singleTileTextureMemory * tilesLength,
                );
                expect(statistics.batchTableByteLength).toEqual(0);

                // One feature colored, the batch table memory is now higher
                tileset.root.content.getFeature(0).color = Color.RED;
                scene.renderForSpecs();
                expect(statistics.geometryByteLength).toEqual(
                  singleTileGeometryMemory * tilesLength,
                );
                expect(statistics.texturesByteLength).toEqual(
                  singleTileTextureMemory * tilesLength,
                );
                expect(statistics.batchTableByteLength).toEqual(
                  singleTileBatchTextureMemory,
                );

                // All tiles picked, the texture memory is now higher
                scene.pickForSpecs();
                expect(statistics.geometryByteLength).toEqual(
                  singleTileGeometryMemory * tilesLength,
                );
                expect(statistics.texturesByteLength).toEqual(
                  singleTileTextureMemory * tilesLength,
                );
                expect(statistics.batchTableByteLength).toEqual(
                  singleTileBatchTextureMemory +
                    singleTilePickTextureMemory * tilesLength,
                );

                // Tiles are still in memory when zoomed out
                viewNothing();
                scene.renderForSpecs();
                expect(statistics.geometryByteLength).toEqual(
                  singleTileGeometryMemory * tilesLength,
                );
                expect(statistics.texturesByteLength).toEqual(
                  singleTileTextureMemory * tilesLength,
                );
                expect(statistics.batchTableByteLength).toEqual(
                  singleTileBatchTextureMemory +
                    singleTilePickTextureMemory * tilesLength,
                );

                // Trim loaded tiles, expect the memory statistics to be 0
                tileset.trimLoadedTiles();
                scene.renderForSpecs();
                expect(statistics.geometryByteLength).toEqual(0);
                expect(statistics.texturesByteLength).toEqual(0);
                expect(statistics.batchTableByteLength).toEqual(0);
              });
            },
          );
        },
      );
    });

(An example that was intentionally chosen to show how subtle that missing return <theProimise> can be, which will likely mess up Jasmine's internal scheduling ...)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Unhandled promise rejection: undefined thrown in CI
4 participants