Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(frontend): MkImgWithBlurhashでblurhash描画に使うcanvasは再利用するようにする #10966

Merged
merged 6 commits into from
Jul 2, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 32 additions & 19 deletions packages/frontend/src/components/MkImgWithBlurhash.vue
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,13 @@ import TestWebGL2 from '@/workers/test-webgl2?worker';
import { WorkerMultiDispatch } from '@/scripts/worker-multi-dispatch';
import { extractAvgColorFromBlurhash } from '@/scripts/extract-avg-color-from-blurhash';

const workerPromise = new Promise<WorkerMultiDispatch | null>(resolve => {
const canvasPromise = new Promise<WorkerMultiDispatch | HTMLCanvasElement>(resolve => {
// テスト環境で Web Worker インスタンスは作成できない
if (import.meta.env.MODE === 'test') {
resolve(null);
const canvas = document.createElement('canvas');
canvas.width = 64;
canvas.height = 64;
Comment on lines +29 to +30
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

そういえばこれもっと大きくても良いのでは

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

そう?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

サイズが大きくても buraha の速度はほぼ変わらない

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

保持しておくcanvasのサイズが大きいとメモリを食いまくるような気がする

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(表示に使うHTMLCanvasElementの話

resolve(canvas);
return;
}
const testWorker = new TestWebGL2();
Expand All @@ -38,7 +41,10 @@ const workerPromise = new Promise<WorkerMultiDispatch | null>(resolve => {
resolve(workers);
if (_DEV_) console.log('WebGL2 in worker is supported!');
} else {
resolve(null);
const canvas = document.createElement('canvas');
canvas.width = 64;
canvas.height = 64;
resolve(canvas);
if (_DEV_) console.log('WebGL2 in worker is not supported...');
}
testWorker.terminate();
Expand Down Expand Up @@ -70,6 +76,7 @@ const props = withDefaults(defineProps<{
width?: number;
cover?: boolean;
forceBlurhash?: boolean;
onlyAvgColor?: boolean; // 軽量化のためにBlurhashを使わずに平均色だけを描画
}>(), {
transition: null,
src: null,
Expand All @@ -79,6 +86,7 @@ const props = withDefaults(defineProps<{
width: 64,
cover: true,
forceBlurhash: false,
onlyAvgColor: false,
});

const viewId = uuid();
Expand Down Expand Up @@ -139,8 +147,8 @@ function drawImage(bitmap: CanvasImageSource) {
ctx.drawImage(bitmap, 0, 0, canvasWidth, canvasHeight);
}

async function draw() {
if (!canvas.value || props.hash == null) return;
function drawAvg() {
if (!canvas.value || !props.hash) return;

const ctx = canvas.value.getContext('2d');
if (!ctx) return;
Expand All @@ -149,25 +157,28 @@ async function draw() {
ctx.beginPath();
ctx.fillStyle = extractAvgColorFromBlurhash(props.hash) ?? '#888';
ctx.fillRect(0, 0, canvasWidth, canvasHeight);
}

async function draw() {
if (props.hash == null) return;

const workers = await workerPromise;
if (workers) {
workers.postMessage(
drawAvg();

if (props.onlyAvgColor) return;

const work = await canvasPromise;
if (work instanceof WorkerMultiDispatch) {
work.postMessage(
{
id: viewId,
hash: props.hash,
width: canvasWidth,
height: canvasHeight,
},
undefined,
);
} else {
try {
const work = document.createElement('canvas');
work.width = canvasWidth;
work.height = canvasHeight;
render(props.hash, work);
ctx.drawImage(work, 0, 0, canvasWidth, canvasHeight);
drawImage(work);
} catch (error) {
console.error('Error occured during drawing blurhash', error);
}
Expand All @@ -179,9 +190,9 @@ function workerOnMessage(event: MessageEvent) {
drawImage(event.data.bitmap as ImageBitmap);
}

workerPromise.then(worker => {
if (worker) {
worker.addListener(workerOnMessage);
canvasPromise.then(work => {
if (work instanceof WorkerMultiDispatch) {
work.addListener(workerOnMessage);
}

draw();
Expand All @@ -204,8 +215,10 @@ onMounted(() => {
});

onUnmounted(() => {
workerPromise.then(worker => {
worker?.removeListener(workerOnMessage);
canvasPromise.then(work => {
if (work instanceof WorkerMultiDispatch) {
work.removeListener(workerOnMessage);
}
});
});
</script>
Expand Down
3 changes: 2 additions & 1 deletion packages/frontend/src/components/global/MkAvatar.vue
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<template>
<component :is="link ? MkA : 'span'" v-user-preview="preview ? user.id : undefined" v-bind="bound" class="_noSelect" :class="[$style.root, { [$style.animation]: animation, [$style.cat]: user.isCat, [$style.square]: squareAvatars }]" :style="{ color }" :title="acct(user)" @click="onClick">
<img :class="$style.inner" :src="url" :hash="user?.avatarBlurhash" :cover="true"/>
<MkImgWithBlurhash :class="$style.inner" :src="url" :hash="user?.avatarBlurhash" :cover="true" :onlyAvgColor="true"/>
<MkUserOnlineIndicator v-if="indicator" :class="$style.indicator" :user="user"/>
<div v-if="user.isCat" :class="[$style.ears]">
<div :class="$style.earLeft">
Expand All @@ -24,6 +24,7 @@
<script lang="ts" setup>
import { watch } from 'vue';
import * as misskey from 'misskey-js';
import MkImgWithBlurhash from '../MkImgWithBlurhash.vue';
import MkA from './MkA.vue';
import { getStaticImageUrl } from '@/scripts/media-proxy';
import { extractAvgColorFromBlurhash } from '@/scripts/extract-avg-color-from-blurhash';
Expand Down
8 changes: 5 additions & 3 deletions packages/frontend/src/workers/draw-blurhash.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
import { render } from 'buraha';

const canvas = new OffscreenCanvas(64, 64);

onmessage = (event) => {
// console.log(event.data);
if (!('id' in event.data && typeof event.data.id === 'string')) {
Expand All @@ -8,8 +10,8 @@ onmessage = (event) => {
if (!('hash' in event.data && typeof event.data.hash === 'string')) {
return;
}
const work = new OffscreenCanvas(event.data.width ?? 64, event.data.height ?? 64);
render(event.data.hash, work);
const bitmap = work.transferToImageBitmap();

render(event.data.hash, canvas);
const bitmap = canvas.transferToImageBitmap();
postMessage({ id: event.data.id, bitmap });
};