Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug][Sort] Mongodb2StarRocksTest failure due to potential dependency conflicts #11166

Closed
2 tasks done
PeterZh6 opened this issue Sep 21, 2024 · 0 comments · Fixed by #11170
Closed
2 tasks done

[Bug][Sort] Mongodb2StarRocksTest failure due to potential dependency conflicts #11166

PeterZh6 opened this issue Sep 21, 2024 · 0 comments · Fixed by #11170
Assignees
@PeterZh6
Labels
type/bug Something is wrong
Milestone
1.14.0

Comments

@PeterZh6
Copy link
Contributor

What happened

When running the Mongodb2StarRocksTest in the sort-end-to-end-test-v1.15 module, the test failed due to a NoClassDefFoundError exception caused by a missing org.mongodb.bson module. This exception occurred when mongodb-cdc attempted to connect to MongoDB, but the required BSON dependency was not available in the classpath. The issue appears to be due to a missing or misconfigured dependency for the MongoDB driver in the test environment.

Attached are the screenshots of the problem and the logs.
MongoDBError1
MongoDBError2

[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.inlong.sort.tests.Mongodb2StarRocksTest
ERROR StatusLogger Reconfiguration failed: No configuration found for '7f31245a' at 'null' in 'null'
11:15:26.905 [main] ERROR org.apache.inlong.sort.tests.Mongodb2StarRocksTest -
--------------------------------------------------------------------------------
Test testMongodbUpdateAndDelete(org.apache.inlong.sort.tests.Mongodb2StarRocksTest) failed with:
java.lang.AssertionError: expected:<[2,tom,Big 2-wheel scooter ]> but was:<[]>
        at org.junit.Assert.fail(Assert.java:89)
        at org.junit.Assert.failNotEquals(Assert.java:835)
        at org.junit.Assert.assertEquals(Assert.java:120)
        at org.junit.Assert.assertEquals(Assert.java:146)
        at org.apache.inlong.sort.tests.utils.JdbcProxy.checkResult(JdbcProxy.java:80)
        at org.apache.inlong.sort.tests.utils.JdbcProxy.checkResultWithTimeout(JdbcProxy.java:99)
        at org.apache.inlong.sort.tests.Mongodb2StarRocksTest.testMongodbUpdateAndDelete(Mongodb2StarRocksTest.java:147)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
        at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
        at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
        at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
        at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
        at org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45)
        at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
        at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
        at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
        at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
        at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
        at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
        at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
        at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
        at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
        at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
        at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
        at org.testcontainers.containers.FailureDetectingExternalResource$1.evaluate(FailureDetectingExternalResource.java:30)
        at org.testcontainers.containers.FailureDetectingExternalResource$1.evaluate(FailureDetectingExternalResource.java:30)
        at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
        at org.junit.rules.RunRules.evaluate(RunRules.java:20)
        at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
        at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
        at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
        at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
        at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42)
        at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80)
        at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:107)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
        at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114)
        at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86)
        at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86)
        at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:55)
        at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:223)
        at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:175)
        at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:139)
        at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:456)
        at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:169)
        at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:595)
        at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:581)

================================================================================
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 135.708 s <<< FAILURE! - in org.apache.inlong.sort.tests.Mongodb2StarRocksTest
[ERROR] org.apache.inlong.sort.tests.Mongodb2StarRocksTest.testMongodbUpdateAndDelete  Time elapsed: 109.679 s  <<< FAILURE!
java.lang.AssertionError: expected:<[2,tom,Big 2-wheel scooter ]> but was:<[]>
        at org.junit.Assert.fail(Assert.java:89)
        at org.junit.Assert.failNotEquals(Assert.java:835)
        at org.junit.Assert.assertEquals(Assert.java:120)
        at org.junit.Assert.assertEquals(Assert.java:146)
        at org.apache.inlong.sort.tests.utils.JdbcProxy.checkResult(JdbcProxy.java:80)
        at org.apache.inlong.sort.tests.utils.JdbcProxy.checkResultWithTimeout(JdbcProxy.java:99)
        at org.apache.inlong.sort.tests.Mongodb2StarRocksTest.testMongodbUpdateAndDelete(Mongodb2StarRocksTest.java:147)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
        at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
        at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
        at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
        at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
        at org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45)
        at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
        at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
        at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
        at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
        at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
        at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
        at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
        at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
        at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
        at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
        at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
        at org.testcontainers.containers.FailureDetectingExternalResource$1.evaluate(FailureDetectingExternalResource.java:30)
        at org.testcontainers.containers.FailureDetectingExternalResource$1.evaluate(FailureDetectingExternalResource.java:30)
        at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
        at org.junit.rules.RunRules.evaluate(RunRules.java:20)
        at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
        at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
        at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
        at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
        at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42)
        at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80)
        at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:107)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
        at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114)
        at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86)
        at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86)
        at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:55)
        at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:223)
        at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:175)
        at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:139)
        at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:456)
        at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:169)
        at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:595)
        at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:581)

[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]   Mongodb2StarRocksTest.testMongodbUpdateAndDelete:147 expected:<[2,tom,Big 2-wheel scooter ]> but was:<[]>
[INFO]
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------

What you expected to happen

The Mongodb2StarRocksTest should have passed without throwing any exceptions because the MongoDB driver had been properly added to the mongodb-cdc dependency. The test was expected to run successfully, verifying MongoDB data sync operations with StarRocks.

Attached is the screenshot of a successful test result.
image

How to reproduce

To reproduce the issue:

  1. Pull the latest InLong code from GitHub.
  2. Run the Mongodb2StarRocksTest in the sort-end-to-end-test-v1.15 module.
  3. To observe the full stack trace of the exception, you may need to modify the test to prevent the containers from being torn down immediately after failure. This can be achieved by adding an infinite loop (while(true){}) at the end of the test to keep the containers alive, allowing inspection of the exception details.

Test Modification:

In the Mongodb2StarRocksTest, modify the testMongodbUpdateAndDelete method as shown below to prevent the test from ending prematurely:
More specifically, change testMongodbUpdateAndDelete of Mongodb2StarRocksTest in this manner

@Test
public void testMongodbUpdateAndDelete() throws Exception {
    submitSQLJob(sqlFile, jdbcJar, mongodbJar, mysqlJdbcJar);
    waitUntilJobRunning(Duration.ofSeconds(10));

    // generate input
    MongoClient mongoClient = MongoClients.create(MONGODB_CONTAINER.getConnectionString());
    MongoDatabase database = mongoClient.getDatabase("test");
    MongoCollection<Document> collection = database.getCollection("test_input1");

    Document document1 = new Document("_id", 1)
            .append("name", "jacket")
            .append("description", "water resistent white wind breaker");
    Document document2 = new Document("_id", 2)
            .append("name", "scooter")
            .append("description", "Big 2-wheel scooter ");
    List<Document> documents = new ArrayList<Document>();
    documents.add(document1);
    documents.add(document2);
    collection.insertMany(documents);
    collection.updateOne(eq("_id", 2), combine(set("name", "tom")));
    collection.deleteOne(eq("_id", 1));

    JdbcProxy proxy =
            new JdbcProxy(STAR_ROCKS.getJdbcUrl(), STAR_ROCKS.getUsername(),
                    STAR_ROCKS.getPassword(),
                    STAR_ROCKS.getDriverClassName());
    List<String> expectResult =
            Arrays.asList("2,tom,Big 2-wheel scooter ");

    // Commented out the result check to keep containers alive
    // proxy.checkResultWithTimeout(expectResult, "test_output1", 3, 60000L);

    // Infinite loop to prevent teardown
    while(true) {
    }
}

Environment

WSL Ubuntu-22.04
JDK 8
Maven 3.8.8

InLong version

master

InLong Component

InLong Sort

Are you willing to submit PR?

  • Yes, I am willing to submit a PR!

Code of Conduct
@PeterZh6 PeterZh6 added the type/bug Something is wrong label Sep 21, 2024
This was referenced Sep 21, 2024
Merged
Merged
Merged
@aloyszhang aloyszhang added this to the 1.14.0 milestone Sep 22, 2024
@PeterZh6 PeterZh6 changed the title [Bug][Sort] Mongodb2StarRocksTest Failure Due to Potential Dependency Conflicts [Bug][Sort] Mongodb2StarRocksTest failure due to potential dependency conflicts Sep 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@PeterZh6 PeterZh6

Labels
type/bug Something is wrong
Projects
None yet
Milestone
1.14.0
2 participants
@aloyszhang @PeterZh6