The r-rspm Dockerfiles from the Rocker Project provides basic building using Ubuntu (as opposed to Debian in r-base) upon which other containers can be built. They also offer (initial, still somewhat experimental) support for the RSPM (RStudio Package Manager) binaries for the supported bionic (aka 18.04) and focal (aka 20.04) releases.
As an alternative consider the the r-ubuntu containers which are also useful due to the large number of packages offered via the PPAs, and their more complete package dependency tracking. However, RSPM offers more binaries.
Documentation is being added at the Wiki. Please file issue tickets at the rocker/ issue tracker.