Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(rendering): migrate chunk mesh generation Flux #4786

Merged
merged 75 commits into from
May 13, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
75 commits
Select commit Hold shift + click to select a range
2723a05
refactor(rendering): rework chunk mesh generation with RXJava
pollend Jun 21, 2021
f0ac410
cleanup
pollend Jun 21, 2021
ca841b7
chore: replace with computation
pollend Jun 21, 2021
093921c
refactor(rendering): rework chunk mesh generation with RXJava Flowable
pollend Jun 23, 2021
7791254
update tessellator
pollend Jun 23, 2021
8016258
chore: cleanup
pollend Jun 25, 2021
dea7815
chore: cleanup
pollend Jun 26, 2021
9df6538
Merge branch 'develop' into refactor/migrate-chunk-mesh-generation-fl…
pollend Jun 27, 2021
a8e2d11
Merge branch 'develop' of github.com:MovingBlocks/Terasology into ref…
pollend Jun 27, 2021
5902bbd
chore: add info
pollend Jun 27, 2021
1456a7a
Merge branch 'refactor/migrate-chunk-mesh-generation-flowable' of git…
pollend Jun 27, 2021
9b04a71
chore: better handle processing chunks that are unloaded
pollend Jun 27, 2021
543a293
chore: replace with reactor
pollend Jul 3, 2021
2826532
Merge branch 'develop' into refactor/migrate-chunk-mesh-generation-fl…
pollend Jul 3, 2021
08cafdc
chore: batch process chunks
pollend Jul 4, 2021
0acab6b
Merge branch 'refactor/migrate-chunk-mesh-generation-flowable' of git…
pollend Jul 4, 2021
d088ace
Merge branch 'develop' into refactor/migrate-chunk-mesh-generation-fl…
pollend Jul 4, 2021
8c6e918
chore: minor cleanup
pollend Jul 4, 2021
3da70fc
Merge branch 'refactor/migrate-chunk-mesh-generation-flowable' of git…
pollend Jul 4, 2021
f53e6bc
chore: limit number of parallel rails
pollend Jul 4, 2021
587dc29
Merge branch 'develop' into refactor/migrate-chunk-mesh-generation-fl…
pollend Jul 5, 2021
8d4264f
Merge branch 'develop' into refactor/migrate-chunk-mesh-generation-fl…
pollend Aug 1, 2021
4a951a8
Merge branch 'develop' into refactor/migrate-chunk-mesh-generation-fl…
pollend Aug 7, 2021
0005fcd
Merge branch 'develop' into refactor/migrate-chunk-mesh-generation-fl…
pollend Aug 24, 2021
f2ba2d7
Merge branch 'develop' of github.com:MovingBlocks/Terasology into ref…
pollend Aug 28, 2021
b04e07e
Merge branch 'refactor/migrate-chunk-mesh-generation-flowable' of git…
pollend Aug 28, 2021
bad0581
Merge branch 'develop' of github.com:MovingBlocks/Terasology into ref…
pollend Aug 28, 2021
fef81c6
chore: remove duplicate reactor and scheulders
pollend Aug 29, 2021
cfc7d58
chore: update interfaces
pollend Aug 29, 2021
7cbdc32
Merge branch 'develop' into refactor/migrate-chunk-mesh-generation-fl…
pollend Sep 3, 2021
73085f8
chore: clean up RenderableWorldImpl
pollend Sep 3, 2021
a1e3ae1
Merge branch 'refactor/migrate-chunk-mesh-generation-flowable' of git…
pollend Sep 3, 2021
79f80f6
chore: remove ChunkTask
pollend Sep 3, 2021
262c677
Merge remote-tracking branch 'origin/develop' into refactor/migrate-c…
keturn Sep 5, 2021
bd783e6
Merge branch 'develop' into refactor/migrate-chunk-mesh-generation-fl…
pollend Sep 5, 2021
e15b975
chore: simplify consumer
pollend Sep 5, 2021
950acbe
handle emission error when submitting chunk for mesh
pollend Sep 6, 2021
e5069c7
Merge branch 'develop' into refactor/migrate-chunk-mesh-generation-fl…
pollend Oct 3, 2021
de2835d
Merge branch 'develop' into refactor/migrate-chunk-mesh-generation-fl…
pollend Nov 7, 2021
7072611
Merge branch 'develop' of github.com:MovingBlocks/Terasology into ref…
pollend Nov 20, 2021
60a63fe
Merge branch 'develop' into refactor/migrate-chunk-mesh-generation-fl…
pollend Nov 27, 2021
25e2983
Merge branch 'develop' into refactor/migrate-chunk-mesh-generation-fl…
pollend Dec 5, 2021
1bfcd3c
refactor: split out chunk work as a seperate helper class
pollend Dec 6, 2021
e7663be
chore: cleanup world renderer
pollend Dec 10, 2021
93fe4f8
Merge remote-tracking branch 'origin/develop' into refactor/migrate-c…
keturn Jan 19, 2022
4667639
fix(RenderableWorldImpl): update for ChunkMonitor change #4888
keturn Jan 19, 2022
90791df
Merge branch 'refactor/migrate-chunk-mesh-generation-flowable' into r…
keturn Jan 19, 2022
a15466b
chore(rendering.world): lint
keturn Jan 19, 2022
8e227a1
Merge pull request #4972 from MovingBlocks/refactor/split-chunk-proce…
keturn Jan 19, 2022
9562e86
test(ChunkMeshWorker): initial sketch of tests
keturn Jan 20, 2022
ff3b09c
refactor(Chunk): provide default implementations of methods where pos…
keturn Jan 20, 2022
130dfcd
refactor(ChunkMeshWorker): factor out the generateMesh method
keturn Jan 20, 2022
e032131
test(DummyChunk): stub implementation of Chunk
keturn Jan 20, 2022
e4c723a
test(ChunkMeshWorker): initial attempt at using reactor-test
keturn Jan 20, 2022
e9e32b7
test(ChunkMeshWorker): implement more tests
keturn Jan 22, 2022
cf27ab9
Merge remote-tracking branch 'origin/develop' into refactor/migrate-c…
keturn Jan 22, 2022
3f5a78f
Merge branch 'refactor/migrate-chunk-mesh-generation-flowable' into t…
keturn Jan 22, 2022
cf7710a
fix(ChunkMeshWorker): fix no-longer-processing test
keturn Jan 22, 2022
095e40d
refactor(ChunkMeshWorker): inline fluxNewMeshes
keturn Jan 22, 2022
72b0f67
refactor(Chunk): push dispose-old-mesh logic down in to Chunk.setMesh
keturn Jan 22, 2022
ca2f3be
refactor(ChunkMeshWorker): make worker function definitions more compact
keturn Feb 6, 2022
b41ebb1
Merge branch 'develop' into refactor/migrate-chunk-mesh-generation-fl…
keturn Feb 6, 2022
761a492
Merge branch 'develop' into refactor/migrate-chunk-mesh-generation-fl…
keturn Feb 18, 2022
430bafa
Merge remote-tracking branch 'origin/refactor/migrate-chunk-mesh-gene…
keturn Feb 18, 2022
972ee97
test(ChunkMeshWorker): remove some difficult to implement tests about…
keturn Feb 18, 2022
1f4ac45
Merge branch 'develop' into refactor/migrate-chunk-mesh-generation-fl…
jdrueckert Apr 3, 2022
6e534d7
Merge remote-tracking branch 'origin/develop' into refactor/migrate-c…
keturn May 3, 2022
4cc9ca1
Merge branch 'refactor/migrate-chunk-mesh-generation-flowable' into t…
keturn May 3, 2022
b313fb0
test(ChunkMeshWorker): make DummyChunk strings slightly more concise
keturn May 6, 2022
3db427f
test(ChunkMeshWorker): fix testMultipleChunks so the results are orde…
keturn May 6, 2022
d69eef8
test(ChunkMeshWorker): add some documentation
keturn May 7, 2022
a157225
Merge remote-tracking branch 'origin/develop' into refactor/migrate-c…
keturn May 7, 2022
f9c6475
Merge branch 'refactor/migrate-chunk-mesh-generation-flowable' into t…
keturn May 7, 2022
cfb307c
Merge pull request #4987 from MovingBlocks/test/chunkmeshworker
keturn May 7, 2022
4c296fe
Merge branch 'develop' into refactor/migrate-chunk-mesh-generation-fl…
keturn May 9, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions engine-tests/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,8 @@ dependencies {

testImplementation('com.google.truth:truth:1.1.3')
testImplementation('com.google.truth.extensions:truth-java8-extension:1.1.3')

implementation("io.projectreactor:reactor-test:3.4.14")
}

task copyResourcesToClasses(type:Copy) {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,246 @@
// Copyright 2022 The Terasology Foundation
// SPDX-License-Identifier: Apache-2.0

package org.terasology.engine.rendering.world;

import org.joml.Vector3i;
import org.joml.Vector3ic;
import org.junit.jupiter.api.Disabled;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.junit.jupiter.MockitoExtension;
import org.terasology.engine.rendering.primitives.ChunkMesh;
import org.terasology.engine.world.chunks.Chunk;
import org.terasology.engine.world.chunks.RenderableChunk;
import reactor.core.publisher.Mono;
import reactor.core.scheduler.Schedulers;
import reactor.test.StepVerifier;
import reactor.test.scheduler.VirtualTimeScheduler;
import reactor.test.subscriber.TestSubscriber;
import reactor.util.function.Tuple2;
import reactor.util.function.Tuples;

import java.time.Duration;
import java.util.Comparator;
import java.util.List;
import java.util.function.Consumer;

import static com.google.common.truth.Truth.assertThat;
import static org.junit.jupiter.api.Assertions.fail;
import static org.mockito.Mockito.mock;

@ExtendWith(MockitoExtension.class)
public class ChunkMeshWorkerTest {
static final Duration EXPECTED_DURATION = Duration.ofSeconds(4);

static Vector3ic position0 = new Vector3i(123, 456, 789);

final Vector3i currentPosition = new Vector3i(position0);

Comparator<RenderableChunk> comparator = Comparator.comparingDouble(chunk ->
chunk.getRenderPosition().distanceSquared(currentPosition.x, currentPosition.y, currentPosition.z)
);
ChunkMeshWorker worker;

/** Creates a new mock ChunkMesh.
* <p>
* A simple work function for {@link ChunkMeshWorker}.
*/
static Mono<Tuple2<Chunk, ChunkMesh>> alwaysCreateMesh(Chunk chunk) {
chunk.setDirty(false);
return Mono.just(Tuples.of(chunk, mock(ChunkMesh.class)));
}

/**
* Create a new Chunk at this position.
* <p>
* The {@link DummyChunk} is marked {@code ready} and {@code dirty}.
*/
static Chunk newDirtyChunk(Vector3ic position) {
var chunk = new DummyChunk(position);
chunk.markReady();
chunk.setDirty(true);
return chunk;
}

/**
* Creates a new ChunkMeshWorker with a StepVerifier on its output.
* <p>
* Sets {@link #worker} to a new {@link ChunkMeshWorker}.
*
* @return A verifier for {@link ChunkMeshWorker#getCompletedChunks()}.
*/
protected StepVerifier.Step<Chunk> completedChunksStepVerifier() {
StepVerifier.setDefaultTimeout(EXPECTED_DURATION);

// Use virtual time so we don't have to wait around in real time
// to see whether there are more events pending.
// Requires that the schedulers be created _inside_ the withVirtualTime supplier.
return StepVerifier.withVirtualTime(() -> {
worker = new ChunkMeshWorker(
ChunkMeshWorkerTest::alwaysCreateMesh,
comparator,
Schedulers.parallel(),
Schedulers.single()
);
return worker.getCompletedChunks();
});
}

/**
* Get completed Chunks as a list.
* <p>
* Applies the given function to a new {@link ChunkMeshWorker}, and returns the list of completed
* {@link Chunk Chunks}.
* <p>
* Assumes the work will not be delayed by more than {@link #EXPECTED_DURATION}.
*/
protected List<Chunk> getChunksThatResultFrom(Consumer<ChunkMeshWorker> withWorker) {
// TODO: Make a VirtualTimeScheduler JUnit Extension so that we don't have these
// two different ways of creating schedulers for the ChunkMeshWorker.
var scheduler = VirtualTimeScheduler.create();

var workerB = new ChunkMeshWorker(
ChunkMeshWorkerTest::alwaysCreateMesh,
comparator,
scheduler,
scheduler
);

var completed = workerB.getCompletedChunks()
.subscribeWith(TestSubscriber.create());

withWorker.accept(workerB);

// The Worker doesn't mark the flux as complete; it expects it'll still get more work.
// That means we can't collect the the complete flux in to a list.
// Instead, we use TestSubscriber's methods to see what it has output so far.
//
// Other things I have tried here:
// * Adding `.timeout(EXPECTED_DURATION)` to the flux, and waiting for the TimeoutError.
// That works, and perhaps allows for less ambiguity about what is happening, but it
// doesn't seem to be necessary.
//
// * Using `.buffer(EXPECTED_DURATION).next()` instead of TestSubscriber.getReceived.
// Did not work; instead of giving me a buffer containing everything from that window,
// waiting on the result just timed out.
//
// See https://stackoverflow.com/a/72116182/9585 for some notes from the reactor-test author
// about this test scenario.
scheduler.advanceTimeBy(EXPECTED_DURATION);
return completed.getReceivedOnNext();
}

@Test
void testMultipleChunks() {
var chunk1 = newDirtyChunk(position0);
var chunk2 = newDirtyChunk(new Vector3i(position0).add(1, 0, 0));

var resultingChunks = getChunksThatResultFrom(worker -> {
worker.add(chunk1);
worker.add(chunk2);
worker.update();
});

assertThat(resultingChunks).containsExactly(chunk1, chunk2);
}

@Test
void testChunkIsNotProcessedTwice() {
var chunk1 = newDirtyChunk(position0);

completedChunksStepVerifier().then(() -> {
worker.add(chunk1);
worker.add(chunk1); // added twice
worker.update();
})
.expectNextCount(1).as("expect only one result")
.then(() -> {
// adding it again and doing another update should still not change
worker.add(chunk1);
worker.update();
})
.verifyTimeout(EXPECTED_DURATION);
}

@Test
void testChunkIsRegeneratedIfDirty() {
var chunk1 = newDirtyChunk(position0);

completedChunksStepVerifier().then(() -> {
worker.add(chunk1);
worker.update();
})
.expectNext(chunk1).as("initial generation")
.then(() -> {
chunk1.setDirty(true);
worker.update();
})
.expectNext(chunk1).as("regenerating after dirty")
.verifyTimeout(EXPECTED_DURATION);
}

@Test
void testChunkCanBeRemovedBeforeMeshGeneration() {
var chunk = newDirtyChunk(position0);
completedChunksStepVerifier().then(() -> {
worker.add(chunk);
worker.remove(chunk);
worker.update();
})
// chunk was removed, no events expected
.verifyTimeout(EXPECTED_DURATION);
}

@Test
void testDoubleRemoveIsNoProblem() {
var chunk = newDirtyChunk(position0);
completedChunksStepVerifier().then(() -> {
worker.add(chunk);
worker.remove(chunk);
worker.update();
})
.then(() -> {
worker.remove(chunk); // second time calling remove on the same chunk
worker.update();
})
// chunk was removed, no events expected
.verifyTimeout(EXPECTED_DURATION);
}

@Test
void testChunkCanBeRemovedByPosition() {
var chunk = newDirtyChunk(position0);
completedChunksStepVerifier().then(() -> {
worker.add(chunk);
worker.remove(position0);
worker.update();
})
// chunk was removed, no events expected
.verifyTimeout(EXPECTED_DURATION);
}

@Test
void testWorkIsPrioritized() {
var nearChunk = newDirtyChunk(position0);
var farChunk = newDirtyChunk(new Vector3i(position0).add(100, 0, 0));

var completed = getChunksThatResultFrom(worker -> {
worker.add(farChunk);
worker.add(nearChunk);
worker.update();
});
// TODO: this may be flaky due to parallelization.
// Given a scheduler with N threads, we should test it with more than N chunks.
assertThat(completed).containsExactly(nearChunk, farChunk).inOrder();

// TODO: change the state of the comparator
// assert the next one through the gate is the one closest *now*
}

@Test
@Disabled("TODO")
void testWorkerStopsWhenShutDown() {
fail("TODO: add shutdown method");
}
}
Loading