When calling the Widget API using .readRoomEvents just after a page reload, only a subset of events are returned

[ghost]

Steps to reproduce

1. Use the example code below. You can create a new React app, add the matrix-widget-api package and paste the code into App.js.
2. Start the widget with HTTPS=true npm start, add it to a room (I'm using plaintext rooms for testing because of past issues with encrypted rooms) with /addwidget https://localhost:3000 and pin it.
3. Click the Send 100 button and wait 100 seconds. Some patience is needed because of the rate limiting.
4. Reload the page using the browser's refresh button or press F5.
5. Note that only a subset of events are displayed.
6. Now click the Read button. Suddenly all 100 events are there.
7. Open App.js and change the value of MS_TO_WAIT from 0 to 5000.
8. Reload the page again.
9. Note that all events load after a 5 second delay.

import './App.css';
import { useEffect } from 'react';
import { WidgetApi } from 'matrix-widget-api'
import { useState } from 'react'

const { widgetId, parentUrl } = window.location.search
  .slice(1)
  .split('&')
  .reduce((p, e) => {
    const [key, value] = e.split('=').map((e) => decodeURIComponent(e));
    return {
      ...p,
      [key]: value,
    };
  }, {});

const widgetApi = new WidgetApi(widgetId, parentUrl);

['send', 'receive'].forEach((direction) =>
  widgetApi.requestCapability(`org.matrix.msc2762.${direction}.event:net.nordeck.test`)
);

const MS_TO_WAIT = 0;

function App() {
  const [events, setEvents] = useState([]);

  useEffect(() => {
    function handleEvent(e) {
      e.preventDefault();
      widgetApi.transport.reply(e.detail, {});
      setEvents(events => [...events, e.detail.data.content])
    }

    widgetApi.on('action:send_event', handleEvent)
    widgetApi.on('ready', () => setTimeout(() => {
      readAllEvents();
    }, MS_TO_WAIT));

    widgetApi.start();

    return () => {
      widgetApi.off('action:send_event', handleEvent)
      widgetApi.off('ready', readAllEvents)
    }
  }, [])

  async function readAllEvents() {
    const events = (await widgetApi.readRoomEvents('net.nordeck.test', Number.MAX_SAFE_INTEGER)).map(e => e.content)
    setEvents(events)
  }

  async function sendNEvents(n) {
    for (let i = 0; i < n; i++) {
      widgetApi.sendRoomEvent('net.nordeck.test', {})
      await new Promise((r) => setTimeout(r, 1000)); // prevent 'Some of your messages have not been sent'
    }
  }

  return (
    <>
      <h1>{events.length}</h1>
      <button onClick={readAllEvents}>Read</button>
      <button onClick={() => sendNEvents(1)}>Send 1</button>
      <button onClick={() => sendNEvents(100)}>Send 10</button>
      <button onClick={() => sendNEvents(100)}>Send 100</button>
      <pre>
        {JSON.stringify(events, null, 2)}
      </pre>
    </>
  );
}
export default App;

Outcome

What did you expect?

I expected that the Widget API would take care of only reading the events once Element is ready and return me the 100 events that are in the room.

What happened instead?

It seems like Element needs some time before it has all the events. There's no event to tell me when Element is ready, so I have to wait an arbitrary amount of time (such as 5 seconds) before reading events.

It's worth noting the issue also occurs in other cases when Element is reloaded, such as when the user updates Element (on develop.element.io I get update prompts regularly).

Operating system

macOS Monterey Version 12.1

Browser information

Google Chrome Version 97.0.4692.99 (Official Build) (arm64)

URL for webapp

develop.element.io

Application version

Element version: 64242a0-react-29cf22a5212c-js-b07457726bf5, Olm version: 3.2.8

Homeserver

matrix.org

Will you send logs?

Yes

[Fox32]

@dbkr I suggest to close this. The expectation that readRoomEvents returns the full history is wrong, it only returns what Element has currently loaded. Instead widgets should use an approach like matrix-org/matrix-spec-proposals#3869 to make sure to access all events it needs.