Playing with Bun

[myovchev]

NOTE be sure to install bun if you want to follow the code below

Intro

What is Bun?

A Javascript runtime (like Node and Deno), but having built-in Web APIs, bundling, dev server and support for major JS Frameworks. It extends/is built on top of Webkit JavaScriptCore and it's written in ZIG. The project website clearly states that this is an experimental project.

Bun natively implements hundreds of Node.js and Web APIs, including ~90% of Node-API functions (native modules), fs, path, Buffer and more.
source: bun.sh

The project goals:

The goal of Bun is to run most of the worlds JavaScript outside of browsers, bringing performance and complexity enhancements to your future infrastructure, as well as developer productivity through better, simpler tooling.

Environment details

All the testing/benchmarking was performed on an Intel 12 core 12th Gen i7 CPU, 32 GB RAM, M.2 NVMe SSD storage machine with 4g internet connection (when that matters). The document tries to compare the performance of npm against the bun built-in package manager and as a whole the file system operations. It also explores the Bun built-in bundling and Framework support.

The tests were performed in the following environment:

• npm v8.11.0
• node v16.16.0
• bun v0.1.5
• Ubuntu 20.04.4 LTS

Notable Features

TS and JSX as first-class citizens, built-in Framework support (dev environment for React App, NextJS, etc), built-in bundling. The docs are clearly stating that the latter two are features that will soon introduce breaking changes. The Dev server and all kind of (e.g HMR) dev goodies are directly supported by the runtime.

Bun's lock file is binary. This combined with other package management and FS techniques leads to hard to believe performance.

Integrated Discord bot development, including dev server. I find it a very fun feature!

Bundling! A quote from the documentation:

Today, bun is mostly focused on bun.js: the JavaScript runtime.

While you could use bun's bundler & transpiler separately to build for browsers or node, bun doesn't have a minifier or support tree-shaking yet. For production browser builds, you probably should use a tool like esbuild or swc.

Longer-term, bun intends to replace Node.js, Webpack, Babel, yarn, and PostCSS (in production).

The out of the box .env support is awesome. As explained in the Bun docs:

bun run automatically loads environment variables from .env into the shell/task. .env files are loaded with the same priority as the rest of bun, so that means:

1. .env.local is first
2. if ($NODE_ENV === "production") .env.production else .env.development
3. .env

Performance Summary

Detailed information about every performance test are available at the end of this document. The tables use aggregated values from multiple (successful) runs - 3 for package manager, 10 for the rest. Package management results are a bit tricky, I've removed edge results which I identify as "network spikes". It might be worth to perform runs on different network connectivity in the future in order to bring up more insights. The aggregations and ratios are using reasonable double precision.

bun install vs npm install

The final row of the table is showing the win ratio. For example bun(x2.83)/npm(x2.25) means bun is 2.83 times faster, npm uses 2.25 times less CPU which also means the time benefit outperforms the CPU cost.

NOTE: in order to follow exactly the same benchmark flow it is advisable to delete the apps folder (rm -rf apps) before running the package management benchmark scripts. This way any differences in behavior will be nullified (e.g attempt to install deep folder dependencies). You can resurrect the apps later with git checkout apps.

	no cache (s/CPU%)	cache (s/CPU%)	lock & cache (s/CPU%)
bun	28.19/145.67	0.33/96.33	0.31/100.67
npm	79.84/64.67	10.32/178.33	22.91/168.67
win(k)/win(k)	bun(x2.83)/npm(x2.25)	bun(x31.27)/bun(x1.85)	bun(x73.90)/bun(x1.68)

NOTE: Bun is failing a lot of non-cached installs (between 1/10 to 1/20 success rate) with unknown reason. The behavior is one and the same - it hangs indefinitely. The behavior is very similar to recently observed by myself and fellow devs npm v6 (we never found the reason, fixed with switch to npm v8).

File system operations performance

The final row of the table is showing the win ratio. For example bun(x7.47) means bun is 7.47 times faster. No attempt for CPU% detection was made for the related benchmarks. This might be done somewhere in the future.

	write (ms)	copy/unlink (ms)	copy (ms)	read/write (ms)
bun	14.513	29.235	18.729	20.41
node	108.4765	73.2358	73.2358	193.1671
winner(k)	bun(x7.47)	bun(x2.51)	bun(x3.91)	bun(x9.46)

Upcoming features

Some of the most exciting upcoming features (personal opinion):

• Web Streams
• WebSocket server
• More npm compatible package hoisting
• More bundling (tree shaking, minify, source maps, etc)

Bun & Apostrophe

There is a long road ahead in terms of full Node compatibility when it comes to the Bun runtime. I made an attempt to run the Apostrophe app and a build task driven only by curiosity, I had zero expectations. The app boot attempt:

• Apostrophe entry point fail due to "cluster" undefined - not supported, not a big deal. It's not a popular core feature, so I've monkey patched the core and completely removed it.
• A strange error related with the package.json version check, but as far as I remember, it involved runtime check. Monkey patched.
• Error in the entry point traced down to Lodash' deep merge ending at their "base" API. I've replaced the merge call with simple mock objects to be sure Lodash is the reason of the error. Confirmed.
• End of the attempt

Honestly I was surprised that Lodash library failed with the Bun runtime. In the end it is an universal package. However in Bun's defense, there is a good chance that Lodash is doing weird stuff as a price of the "universality" that could be runtime checks, which I may assume is not working well (this could raise a discussion related to a call for a standard runtime API now that we are approaching Node, Deno, Bun...?). However, I haven't dig into it, it's just a wild guess. As a whole this was a naive try, because Bun's roadmap mentions upcoming Express support (whatever that means) and I really doubt mongodb client is fully supported (haven't tried it though).

Running the build asset task ended up pretty quick. It just hangs forever with no error. After the above findings I just left it there.

Beside that in relative short & long term:

• Bundling could become very interesting topic giving there is compatible to Webpack ecosystem (and this is the promise made in the docs, see above). The performance boost would be huge.
• Local development and staging/production installs (package management). Once it's fully compatible with NPM it would be a win (less dev and CI times)
• All kind of (Filesystem) tooling and provisioning! The FS operations performance is exceptional. I can see Bun working well together with Bun dedicated TS scripting in e.g. GitHub Action container
• I wonder what would be the benefit (in terms of testing time win) of being able to run Apostrophe UI tests with Bun runtime once/if this is possible (yeah, looking at you Cypress).

Conclusion

Bun is a very young but extremely ambitious and promising project. I would say "watch this space - bun.sh". My only concern is that it's too ambitious. And this is in a no way a critique but a real life lesson, learned the hard way. The architecture at first glance (just what is obvious from the project documentation) addresses the complexity related with the all moving parts/features. Adding the fact that it's built on mostly unpopular low level language (opinionated, could be just my own ignorance), the project future is tied to the amount of community and financial support it can get and probably the big tech reaction. As a strong open source believer that I am, I rate the former much higher.

Full Node compatibility is a key for success in my opinion. I have found already different default behavior for the copyFile API. Such problems will be a great pain to hunt, fix and support in long term. And even the more reasonable question would be "what is the right default behavior". Could this bring us (the developers) back to the point we need to adapt our code for different runtimes (thinking about IE times here)?

Package Install (no cache)

bun install

Available as ./bun-no-cache in this repository.

# Clear the install cache
rm -rf ~/.bun/install/cache && \
rm -f package-lock.json && \
rm -f bun.lockb && \
rm -rf node_modules/

# Benchmark
\time -f "Total time: %E\nCPU: %P" bun install

Results (4g connection):

Total time: 0:29.32
CPU: 155%

Total time: 0:29.56
CPU: 144%

Total time: 0:25.71
CPU: 138%

NOTE: it seems that most of the time Bun is hanging and it's not finishing the install (due to network spikes is my wild guess) . This happens mostly at the end of the install (e.g. at [3873/3875]). The results here include only the successful runs.

npm install

Available as ./npm-no-cache in this repository.

# Clear the install cache
rm -rf ./data/npm-cache  && \
rm -f package-lock.json && \
rm -rf node_modules/ && \
npm cache verify --cache ./data/npm-cache

# Benchmark
\time -f "\nTotal time: %E\nCPU: %P" npm install --cache ./data/npm-cache   

Results (4g connection):

Total time: 1:17.80
CPU: 64%

Total time: 1:14.49
CPU: 71%

Total time: 1:26.91
CPU: 59%

Package Install (w/ package manager cache, w/o package lock)

bun install

Available as ./bun-no-lock in this repository.

# Ensure caching
bun install

# Clear the node_modules and the lock
du -sh ~/.bun/install/cache && \
rm -f bun.lockb \
rm -f package-lock.json \
rm -rf node_modules/

# Benchmark
\time -f "Total time: %E\nCPU: %P" bun install

Results (4g connection):

Total time: 0:00.33
CPU: 97%

Total time: 0:00.33
CPU: 95%

Total time: 0:00.33
CPU: 97%

NOTE: not sure what's the reason behind the huge difference between some first (7-8 seconds) and all consequent runs of this script. Recording the consistent (without those picks).

npm install

Available as ./npm-no-lock in this repository.

# Ensure caching
npm i

# Clear the node_modules and the lock
rm -f package-lock.json && \
rm -rf node_modules/ && \
npm cache verify --cache ./data/npm-cache

# Benchmark
\time -f "\nTotal time: %E\nCPU: %P" npm install --cache ./data/npm-cache   

Results (4g connection):

Total time: 0:10.35
CPU: 178%

Total time: 0:10.12
CPU: 179%

Total time: 0:10.48
CPU: 178%

Package Install (w/ package manager cache and lock)

bun install

Available as ./bun-cache in this repository.

# Ensure caching
bun install

# Clear the node_modules only
du -sh ~/.bun/install/cache && \
du -sh bun.lockb && \
rm -f package-lock.json \
rm -rf node_modules/

# Benchmark
\time -f "Total time: %E\nCPU: %P" bun install

Results (4g connection):

Total time: 0:00.31
CPU: 102%

Total time: 0:00.32
CPU: 100%

Total time: 0:00.31
CPU: 100%

NOTE: I had to double check my cleanup script to ensure the results are correct. The time looks like messed up benchmark. It's not. Fun fact: \time -f "Total time: %E\nCPU: %P" cp -ar node_modules node_modules2 (the native linux copy recursive command) outputs:

Total time: 0:05.36
CPU: 65%

npm install

Available as ./npm-cache in this repository.

# Ensure caching
npm i

# Clear the node_modules only
du -sh package-lock.json && \
rm -rf node_modules/ && \
npm cache verify --cache ./data/npm-cache

# Benchmark
\time -f "\nTotal time: %E\nCPU: %P" npm install --cache ./data/npm-cache   

Results (4g connection):

Total time: 0:26.98
CPU: 151%

Total time: 0:20.87
CPU: 177%

Total time: 0:20.87
CPU: 178%

Write thousand times a file

Node

Execute 10 times JS script that is writing 1000 times in a file

for i in {1..10}; do node write-thousand-times.mjs; done

Result:

1000 times: 118.115ms
1000 times: 104.312ms
1000 times: 117.269ms
1000 times: 102.417ms
1000 times: 111.279ms
1000 times: 108.314ms
1000 times: 109.583ms
1000 times: 105.929ms
1000 times: 106.473ms
1000 times: 101.074ms

Bun

Execute 10 times JS script that is writing 1000 times in a file.

for i in {1..10}; do bun run write-thousand-times.mjs; done

Result:

[18.19ms] 1000 times
[13.32ms] 1000 times
[12.99ms] 1000 times
[12.03ms] 1000 times
[12.94ms] 1000 times
[21.74ms] 1000 times
[12.64ms] 1000 times
[16.04ms] 1000 times
[14.06ms] 1000 times
[11.18ms] 1000 times

Copy thousand times a file

Note: Bun has different default behavior of copyFile than node 16. Node overwrites existing file, Bun is failing with a permission exception. If this is a desired behavior is not clear at this point. We are unlinking the file after every copy which makes the FS operation even more intense.

Node

Execute 10 times JS script that performs copy/unlink of the same file 1000 times. The source is available in this repository.

for i in {1..10}; do node copy-thousand-times.mjs; done

Result:

1000 times: 124.099ms
1000 times: 82.157ms
1000 times: 87.65ms
1000 times: 114.204ms
1000 times: 103.285ms
1000 times: 88.966ms
1000 times: 103.97ms
1000 times: 106.35ms
1000 times: 132.394ms
1000 times: 105.227ms

Execute 10 times JS script that performs copy to a different file 1000 times. The source is available in this repository.

for i in {1..10}; do node copy-thousand-times-2.mjs; done

Result:

1000 times: 96.422ms
1000 times: 86.679ms
1000 times: 67.656ms
1000 times: 68.466ms
1000 times: 97.386ms
1000 times: 58.561ms
1000 times: 59.238ms
1000 times: 74.225ms
1000 times: 61.898ms
1000 times: 61.827ms

Bun

Execute 10 times JS script that performs copy/unlink of the same file 1000 times. The source is available in this repository.

for i in {1..10}; do bun run copy-thousand-times.mjs; done

Result:

[35.66ms] 1000 times
[24.80ms] 1000 times
[32.68ms] 1000 times
[31.93ms] 1000 times
[29.85ms] 1000 times
[31.25ms] 1000 times
[23.97ms] 1000 times
[33.72ms] 1000 times
[31.61ms] 1000 times
[26.88ms] 1000 times

Execute 10 times (the same) TS script that performs copy/unlink of the same file 1000 times. The source is available in this repository.

for i in {1..10}; do bun run ts/copy-thousand-times.mts; done

Result:

[35.18ms] 1000 times
[32.85ms] 1000 times
[33.45ms] 1000 times
[23.76ms] 1000 times
[24.23ms] 1000 times
[32.54ms] 1000 times
[29.96ms] 1000 times
[35.27ms] 1000 times
[30.96ms] 1000 times
[24.37ms] 1000 times

Execute 10 times JS script that performs copy to a different file 1000 times. The source is available in this repository.

for i in {1..10}; do bun run copy-thousand-times-2.mjs; done

Result:

[16.21ms] 1000 times
[17.90ms] 1000 times
[17.29ms] 1000 times
[22.09ms] 1000 times
[17.78ms] 1000 times
[16.48ms] 1000 times
[16.87ms] 1000 times
[16.97ms] 1000 times
[21.68ms] 1000 times
[24.02ms] 1000 times

Copy thousand times a file content

In this benchmark there are two scripts - node-copy-content-thousand-times.mjs and bun-copy-content-thousand-times.mjs - doing one and the same thing, read file and copy its content to another file. The bun script is using the Bun fast Web API Bun.read and Bun.write

Node

Execute 10 times JS script that performs read content of a file and write it to another file 1000 times. The source is available in this repository.

for i in {1..10}; do node node-copy-content-thousand-times.mjs; done

Result:

1000 times: 217.871ms
1000 times: 199.857ms
1000 times: 195.927ms
1000 times: 202.508ms
1000 times: 183.945ms
1000 times: 182.792ms
1000 times: 189.318ms
1000 times: 192.868ms
1000 times: 178.419ms
1000 times: 188.166ms

Bun

Execute 10 times JS script that performs read content of a file and write it to another file via the Bun Web API 1000 times. The source is available in this repository.

for i in {1..10}; do bun run bun-copy-content-thousand-times.mjs; done

Result:

[54.23ms] 1000 times
[68.34ms] 1000 times
[60.08ms] 1000 times
[56.41ms] 1000 times
[54.49ms] 1000 times
[60.39ms] 1000 times
[46.83ms] 1000 times
[55.68ms] 1000 times
[45.29ms] 1000 times
[50.80ms] 1000 times

Execute 10 times JS script that performs read content of a file and write it to another file via the Node File API 1000 times. The source is available in this repository.

for i in {1..10}; do bun run node-copy-content-thousand-times.mjs; done

Result:

[19.55ms] 1000 times
[21.74ms] 1000 times
[15.75ms] 1000 times
[14.50ms] 1000 times
[20.59ms] 1000 times
[21.24ms] 1000 times
[20.26ms] 1000 times
[20.83ms] 1000 times
[20.11ms] 1000 times
[29.80ms] 1000 times

Frameworks

NextJS (create)

The application created with bun create can be found in apps/nextjs. It was created with the following command:

cd apps && bun create next ./nextjs

It's a flawless development experience. The installation process takes few seconds, bun dev greets you with a high performance development server having all expected features.

NextJS (existing)

This application found in apps/nextjs-demo is an exact copy of the original NextJS demo. It has been "wired" with the following commands (available in the Bun's documentation):

bun add bun-framework-next
echo "framework = 'next'" > bunfig.toml
bun bun # bundle dependencies
bun dev # start dev server

npm run dev (initial run)

• initially - 1682 ms compilation time
• request - 1482 ms compilation time

npm run dev (consequent runs)

• initially - 331 ms compilation time
• request - 440 ms compilation time

bun dev (initial run)

• initially - 1.09 ms compilation time
• request - 133.13 ms compilation time

bun dev (consequent runs)

• initially - 1.09 ms compilation time
• request - 125.46 ms compilation time

Bun is successfully compiling, however the app is broken - the index is double rendered, the routing throws error. Bun is also giving tracing information. I'm not attempting to fix as this is meant to be a compatibility test. It's worth mentioning the demo is exploring server side rendering and partial server rendering which might be the reason for the failure.

ReactJS (create)

Application apps/reactjs is created via bun create react ./reactjs. Same as NextJS create command - flawless experience. The initial build time is 1.04 ms (bun dev). There are no package scripts exposed for starting the app.

ReactJS (existing)

The app apps/reactjs-todo is created via npx create-react-app reactjs-todo. The TODO app is directly copied from this tutorial
Additional commands:

bun add -d react-refresh
bun bun ./src/index.js
bun dev

Bun is successfully compiling, however the app is broken (white screen, no errors). npm start is no more working for this app (as it was before integrating with bun).