feat: use nightly releases of js-ceramic packages

[smrz2001]

No description provided.

[linear]

WS1-1458 Use nightly releases in ECS e2e tests

[smrz2001]

This makes the image quite a bit larger but I was wasting time trying to figure out alpine issues so 🤷🏼

Long-term we'll be working in the new tests repo anyway.

[stbrody]

This should probably be CeramicReaderWriter, not Ceramic. In most of the test configs this test is going to be using the http-client, not the core client. I think the only test that should use the Ceramic type is the state store test b/c it has to restart the node and only runs in the local_node variant with the core client.

[smrz2001]

I did it this way because StreamReaderWriter doesn't have a did method:

    const originalDid = ceramic.did as DID

I can change it to get the DID from the client?

    const originalDid = ceramicClient.did as DID

[smrz2001]

I reverted it to Ceramic because I couldn't get a test to pass, and I don't want to spend time debugging 🤷🏼 That should be a separate ticket, IMO, so I opened one.

Error: Can not verify signature for commit bagcqcera4ebtl3n7r6p45wpopizjc3g4tpsxcah7xnen3bokqbjqex54zwgq to stream kjzl6kcym7w8yahoezjzd0bhmriehs2flqirz9eksga9vatmxy8d4tjg9yvuuml which has controller DID did:key:z6Mko9DSC8znmyZbgQQL9u17ed5ogQSiwh24zSRtvwn6NDeT: invalid_jws: not a valid verificationMethod for issuer: did:key:z6MktamTUdbhrA7FbSqj1g3GpQrAuAQRJtySqiZ4sFdbRE9z#z6MktamTUdbhrA7FbSqj1g3GpQrAuAQRJtySqiZ4sFdbRE9z

    at Function.verifyCommitSignature (file:///home/mz/src/ceramic-integration-tests/node_modules/@ceramicnetwork/common/src/utils/signature_utils.ts:54:13)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at ModelInstanceDocumentHandler._applyGenesis (file:///home/mz/src/ceramic-integration-tests/node_modules/@ceramicnetwork/stream-model-instance-handler/src/model-instance-document-handler.ts:121:7)
    at StateManipulator._applyLog (file:///home/mz/src/ceramic-integration-tests/node_modules/@ceramicnetwork/core/src/stream-loading/state-manipulator.ts:69:17)
    at Repository._genesisFromNetwork (file:///home/mz/src/ceramic-integration-tests/node_modules/@ceramicnetwork/core/src/state-management/repository.ts:390:19)
    at file:///home/mz/src/ceramic-integration-tests/node_modules/@ceramicnetwork/core/src/state-management/repository.ts:269:19
    at file:///home/mz/src/ceramic-integration-tests/node_modules/p-queue/dist/index.js:187:36

[smrz2001]

All tests are green now.

[stbrody]

Wait, it failed the test at runtime? I literally don't see how changing a typescript type, which disappears at runtime, could possibly change runtime behavior? I would expect it possibly to fail to compile, but not to change behavior at runtime...

I'd like to look into this more, how can I run the tests to reproduce this?

[stbrody]

Okay I'm assuming the runtime failures were caused by messing with the did to work around compile failures. I put together a branch here that avoids compile failures without doing any messing with the did. Can you check if this works and passes tests, and if so merge into this PR?

#230

[smrz2001]

Wait, it failed the test at runtime? I literally don't see how changing a typescript type, which disappears at runtime, could possibly change runtime behavior? I would expect it possibly to fail to compile, but not to change behavior at runtime...

Yes, exactly. It wasn't the change itself but likely the fact that I had to remove the line that restored the DID in afterAll that resulted in the failure.

I'd like to look into this more, how can I run the tests to reproduce this?

I was running the tests locally and seeing the same errors in CI.

I'll give the PR a shot, thanks.

[smrz2001]

OK, all tests are passing, thanks, @stbrody.

[PaulLeCam]

LGTM, I'm just wondering what is the update process for the Ceramic package versions please?
Is it possible to automatically use the latest nightlies when the tests run?

[smrz2001]

LGTM, I'm just wondering what is the update process for the Ceramic package versions please?
Is it possible to automatically use the latest nightlies when the tests run?

Thanks @PaulLeCam. Yes, good question. The update.sh script will run at the start of every test and update the dependencies to the latest nightlies. So, even if the latest package versions aren't checked in, we'll always be running tests with the latest versions and can check the versions from the logs if something fails. They just won't be committed until someone goes back in and runs the script by hand and commits the updates, but that shouldn't matter much from now on.

[PaulLeCam]

Thanks @PaulLeCam. Yes, good question. The update.sh script will run at the start of every test and update the dependencies to the latest nightlies. So, even if the latest package versions aren't checked in, we'll always be running tests with the latest versions and can check the versions from the logs if something fails. They just won't be committed until someone goes back in and runs the script by hand and commits the updates, but that shouldn't matter much from now on.

Great, thanks!

[smrz2001]

Opened a separate ticket for local_client-public test flakiness.

[stbrody]

LGTM