Duplicate chunks throw an error

[cooper667]

If you setContent with repetitive data on a collection, an error is thrown when trying to get that content again.

 TypeError: Cannot read properties of undefined (reading 'length')
    at reduce (/etebase-js/src/Crypto.ts:36:49)
    at Array.reduce (<anonymous>)
    at concatArrayBuffersArrays (/etebase-js/src/Crypto.ts:36:26)
    ...

It looks like the index list and/or chunks created here are the issue (the indices are pointers at the original chunks array, but the chunks array is mutated here, so the pointers are incorrect?)

The following tests illustrate the issue (large files are fine, small files are fine, but repetitive data (that would result in chunks that would be deduplicated) throws the error

describe.only("chunking files", () => {
  it("Duplicate Chunks", async () => {
    const collectionManager = etebase.getCollectionManager();
    const col = await collectionManager.create(colType, {}, "");

    const buf = randomBytesDeterministic(10 * 1024, new Uint8Array(32)); // 10kb of psuedorandom data
    const content = JSON.stringify([buf, buf, buf, buf]);

    await col.setContent(content);

    await collectionManager.transaction(col);
    const decryptedContent = await col.getContent(); // <---
    /*
  TypeError: Cannot read properties of undefined (reading 'length')
    at reduce (/etebase-js/src/Crypto.ts:36:49)
    at Array.reduce (<anonymous>)
    at concatArrayBuffersArrays (/etebase-js/src/Crypto.ts:36:26)
    ...
  */

    const out = to_string(decryptedContent);
    expect(out).toEqual(content);
  });

  it("Regular Chunks", async () => {
    const collectionManager = etebase.getCollectionManager();
    const col = await collectionManager.create(colType, {}, "");

    const buf = randomBytesDeterministic(100 * 1024, new Uint8Array(32));
    const content = JSON.stringify(buf);

    await col.setContent(content);

    await collectionManager.transaction(col);
    const decryptedContent = await col.getContent();

    const out = to_string(decryptedContent);
    expect(out).toEqual(content);
  });
  
  it("Small file, no chunks", async () => {
    const collectionManager = etebase.getCollectionManager();
    const col = await collectionManager.create(colType, {}, "");

    const buf = randomBytesDeterministic(10, new Uint8Array(32));
    const content = JSON.stringify(buf);

    await col.setContent(content);

    await collectionManager.transaction(col);
    const decryptedContent = await col.getContent();

    const out = to_string(decryptedContent);
    expect(out).toEqual(content);
  });
});

I'll do a PR with a fix shortly

[tasn]

Let me know once the PR is ready and I'll take a look. I wonder if this also affects the Rust code (and thus every other library).

[tasn]

Did you mean to close this?

[cooper667]

No! I didn't think merging the PR into a fork would close it.

PR is #54