Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Doc] add readme for js packages #421

Merged
merged 5 commits into from
Oct 24, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
[中文版](./README_cn.md)

# detect

detect model is used to detect the position of label frame in the image.

<img src="https://img.shields.io/npm/v/@paddle-js-models/detect?color=success" alt="version"> <img src="https://img.shields.io/bundlephobia/min/@paddle-js-models/detect" alt="size"> <img src="https://img.shields.io/npm/dm/@paddle-js-models/detect?color=orange" alt="downloads"> <img src="https://img.shields.io/npm/dt/@paddle-js-models/detect" alt="downloads">

# Usage

```js
import * as det from '@paddle-js-models/detect';

// Load model
await det.load();

// Get label index, confidence and coordinates
const res = await det.detect(img);

res.forEach(item => {
// Get label index
console.log(item[0]);
// Get label confidence
console.log(item[1]);
// Get label left coordinates
console.log(item[2]);
// Get label top coordinates
console.log(item[3]);
// Get label right coordinates
console.log(item[4]);
// Get label bottom coordinates
console.log(item[5]);
});
```

# effect
![img.png](https://user-images.githubusercontent.com/43414102/153805288-80f289bf-ca92-4788-b1dd-44854681a03f.png)

Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
[English](./README.md)

# detect

detect模型用于检测图像中label框选位置。

<img src="https://img.shields.io/npm/v/@paddle-js-models/detect?color=success" alt="version"> <img src="https://img.shields.io/bundlephobia/min/@paddle-js-models/detect" alt="size"> <img src="https://img.shields.io/npm/dm/@paddle-js-models/detect?color=orange" alt="downloads"> <img src="https://img.shields.io/npm/dt/@paddle-js-models/detect" alt="downloads">

# 使用

```js
import * as det from '@paddle-js-models/detect';

// 模型加载
await det.load();

// 获取label对应索引、置信度、检测框选坐标
const res = await det.detect(img);

res.forEach(item => {
// 获取label对应索引
console.log(item[0]);
// 获取label置信度
console.log(item[1]);
// 获取检测框选left顶点
console.log(item[2]);
// 获取检测框选top顶点
console.log(item[3]);
// 获取检测框选right顶点
console.log(item[4]);
// 获取检测框选bottom顶点
console.log(item[5]);
});
```

# 效果
![img.png](https://user-images.githubusercontent.com/43414102/153805288-80f289bf-ca92-4788-b1dd-44854681a03f.png)

Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
[中文版](./README_cn.md)

# Facedetect

Facedetect is used for face detection in image. It provides a simple interface. At the same time, you can use your own model.

<img src="https://img.shields.io/npm/v/@paddle-js-models/facedetect?color=success" alt="version"> <img src="https://img.shields.io/bundlephobia/min/@paddle-js-models/facedetect" alt="size"> <img src="https://img.shields.io/npm/dm/@paddle-js-models/facedetect?color=orange" alt="downloads"> <img src="https://img.shields.io/npm/dt/@paddle-js-models/facedetect" alt="downloads">

# Usage

```js
import { FaceDetector } from '@paddle-js-models/facedetect';

const faceDetector = new FaceDetector();
await faceDetector.init();
// Required parameter:imgEle(HTMLImageElement)
// Optional parameter: shrink, threshold
// Result is face area information. It includes left, top, width, height, confidence
const res = await faceDetector.detect(
imgEle,
{ shrink: 0.4, threshold: 0.6 }
);
```

# Performance
+ **multi small-sized face**
<img width="500" src="https://mms-voice-fe.cdn.bcebos.com/pdmodel/face/detection/pic/small.png"/>

+ **single big-sized face**
<img width="500" src="https://mms-voice-fe.cdn.bcebos.com/pdmodel/face/detection/pic/big.png"/>

# Postprocess
This model has a better recognition effect for small-sized faces, and the image will be shrink before prediction, so it is necessary to transform the prediction output data.
<img width="500" src="https://mms-voice-fe.cdn.bcebos.com/pdmodel/face/detection/pic/example.png"/>
The red line indicates the predicted output result, and the green line indicates the converted result. dx dy fw fh are known parameters.

# Reference
[original model link](https://github.com/PaddlePaddle/PaddleHub/tree/release/v2.2/modules/image/face_detection/pyramidbox_lite_mobile)
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
[English](./README.md)

# Facedetect

Facedetect 实现图像中的人脸检测,提供的接口简单,支持用户传入模型。

<img src="https://img.shields.io/npm/v/@paddle-js-models/facedetect?color=success" alt="version"> <img src="https://img.shields.io/bundlephobia/min/@paddle-js-models/facedetect" alt="size"> <img src="https://img.shields.io/npm/dm/@paddle-js-models/facedetect?color=orange" alt="downloads"> <img src="https://img.shields.io/npm/dt/@paddle-js-models/facedetect" alt="downloads">

# 使用

```js
import { FaceDetector } from '@paddle-js-models/facedetect';

const faceDetector = new FaceDetector();
await faceDetector.init();
// 使用时必传图像元素(HTMLImageElement),支持指定图片缩小比例(shrink)、置信阈值(threshold)
// 结果为人脸区域信息,包括:左侧 left,上部 top,区域宽 width,区域高 height,置信度 confidence
const res = await faceDetector.detect(
imgEle,
{ shrink: 0.4, threshold: 0.6 }
);
```

# 效果
+ **多个小尺寸人脸**
<img width="500" src="https://mms-voice-fe.cdn.bcebos.com/pdmodel/face/detection/pic/small.png"/>

+ **单个大尺寸人脸**
<img width="500" src="https://mms-voice-fe.cdn.bcebos.com/pdmodel/face/detection/pic/big.png"/>

# 数据后处理
此人脸检测模型对小尺寸人脸具有更好的识别效果,图像在预测前会进行缩小,因此需要对预测输出数据进行变换,及为**数据后处理过程**。示意如下:
<img width="500" src="https://mms-voice-fe.cdn.bcebos.com/pdmodel/face/detection/pic/example.png"/>
红线标识的是预测输出结果,绿线标识的是经过转换后的结果,二者变换过程所涉及到的 dx dy fw fh均为已知量。

# 参考
[源模型链接](https://github.com/PaddlePaddle/PaddleHub/tree/release/v2.2/modules/image/face_detection/pyramidbox_lite_mobile)
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
[中文版](./README_cn.md)

# humanseg

A real-time human-segmentation model. You can use it to change background. The output of the model is gray value. Model supplies simple api for users.

Api drawHumanSeg can draw human segmentation with a specified background.
Api blurBackground can draw human segmentation with a blurred origin background.
Api drawMask can draw the background without human.


<img src="https://img.shields.io/npm/v/@paddle-js-models/humanseg?color=success" alt="version"> <img src="https://img.shields.io/bundlephobia/min/@paddle-js-models/humanseg" alt="size"> <img src="https://img.shields.io/npm/dm/@paddle-js-models/humanseg?color=orange" alt="downloads"> <img src="https://img.shields.io/npm/dt/@paddle-js-models/humanseg" alt="downloads">

# Usage

```js

import * as humanseg from '@paddle-js-models/humanseg';

// load humanseg model, use 398x224 shape model, and preheat
await humanseg.load();

// use 288x160 shape model, preheat and predict faster with a little loss of precision
// await humanseg.load(true, true);

// get the gray value [2, 398, 224] or [2, 288, 160];
const { data } = await humanseg.getGrayValue(img);

// background canvas
const back_canvas = document.getElementById('background') as HTMLCanvasElement;

// draw human segmentation
const canvas1 = document.getElementById('back') as HTMLCanvasElement;
humanseg.drawHumanSeg(data, canvas1, back_canvas) ;

// blur background
const canvas2 = document.getElementById('blur') as HTMLCanvasElement;
humanseg.blurBackground(data, canvas2) ;

// draw the mask with background
const canvas3 = document.getElementById('mask') as HTMLCanvasElement;
humanseg.drawMask(data, canvas3, back_canvas);

```

## gpu pipeline

```js

// 引入 humanseg sdk
import * as humanseg from '@paddle-js-models/humanseg/lib/index_gpu';

// load humanseg model, use 398x224 shape model, and preheat
await humanseg.load();

// use 288x160 shape model, preheat and predict faster with a little loss of precision
// await humanseg.load(true, true);


// background canvas
const back_canvas = document.getElementById('background') as HTMLCanvasElement;

// draw human segmentation
const canvas1 = document.getElementById('back') as HTMLCanvasElement;
await humanseg.drawHumanSeg(input, canvas1, back_canvas) ;

// blur background
const canvas2 = document.getElementById('blur') as HTMLCanvasElement;
await humanseg.blurBackground(input, canvas2) ;

// draw the mask with background
const canvas3 = document.getElementById('mask') as HTMLCanvasElement;
await humanseg.drawMask(input, canvas3, back_canvas);

```

# Online experience

image human segmentation:https://paddlejs.baidu.com/humanseg

video-streaming human segmentation:https://paddlejs.baidu.com/humanStream

# Performance

<img width="800" src="https://user-images.githubusercontent.com/10822846/126873788-1e2d4984-274f-45be-8716-2a87ddda8c75.png"/>
<img width="800" src="https://user-images.githubusercontent.com/10822846/126873838-e5b68c9b-279f-4cb4-ae90-6aaaecd06aa4.png"/>


# Used in Video Meeting
<p>
<img width="400" src="https://user-images.githubusercontent.com/10822846/126872499-c3fd680e-a01b-4daa-b0cb-acd3290862bd.gif"/>
<img width="400" src="https://user-images.githubusercontent.com/10822846/126872930-4f4c5c5d-5c51-44fe-b2d6-3f83c4e124bc.png"/>
</p>
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
[English](./README.md)

# 人像分割

实时的人像分割模型。使用者可以用于背景替换。需要使用接口 getGrayValue 获取灰度值。
然后使用接口 drawHumanSeg 绘制分割出来的人像,实现背景替换;使用接口 blurBackground 实现背景虚化;也可以使用 drawMask 接口绘制背景,可以配置参数来获取全黑背景或者原图背景。

<img src="https://img.shields.io/npm/v/@paddle-js-models/humanseg?color=success" alt="version"> <img src="https://img.shields.io/bundlephobia/min/@paddle-js-models/humanseg" alt="size"> <img src="https://img.shields.io/npm/dm/@paddle-js-models/humanseg?color=orange" alt="downloads"> <img src="https://img.shields.io/npm/dt/@paddle-js-models/humanseg" alt="downloads">

# 使用

```js

// 引入 humanseg sdk
import * as humanseg from '@paddle-js-models/humanseg';

// 默认下载 398x224 shape 的模型,默认执行预热
await humanseg.load();

// 指定下载更轻量模型, 该模型 shape 288x160,预测过程会更快,但会有少许精度损失
// await humanseg.load(true, true);


// 获取分割后的像素 alpha 值,大小为 [2, 398, 224] 或者 [2, 288, 160]
const { data } = await humanseg.getGrayValue(img);

// 获取 background canvas
const back_canvas = document.getElementById('background') as HTMLCanvasElement;

// 背景替换, 使用 back_canvas 作为新背景实现背景替换
const canvas1 = document.getElementById('back') as HTMLCanvasElement;
humanseg.drawHumanSeg(data, canvas1, back_canvas) ;

// 背景虚化
const canvas2 = document.getElementById('blur') as HTMLCanvasElement;
humanseg.blurBackground(data, canvas2) ;

// 绘制人型遮罩,在新背景上隐藏人像
const canvas3 = document.getElementById('mask') as HTMLCanvasElement;
humanseg.drawMask(data, canvas3, back_canvas);

```

## gpu pipeline

```js

// 引入 humanseg sdk
import * as humanseg from '@paddle-js-models/humanseg/lib/index_gpu';

// 默认下载 398x224 shape 的模型,默认执行预热
await humanseg.load();

// 指定下载更轻量模型, 该模型 shape 288x160,预测过程会更快,但会有少许精度损失
// await humanseg.load(true, true);


// 获取 background canvas
const back_canvas = document.getElementById('background') as HTMLCanvasElement;

// 背景替换, 使用 back_canvas 作为新背景实现背景替换
const canvas1 = document.getElementById('back') as HTMLCanvasElement;
await humanseg.drawHumanSeg(input, canvas1, back_canvas) ;

// 背景虚化
const canvas2 = document.getElementById('blur') as HTMLCanvasElement;
await humanseg.blurBackground(input, canvas2) ;

// 绘制人型遮罩,在新背景上隐藏人像
const canvas3 = document.getElementById('mask') as HTMLCanvasElement;
await humanseg.drawMask(input, canvas3, back_canvas);

```

# 在线体验

图片人像分割:https://paddlejs.baidu.com/humanseg

基于视频流人像分割:https://paddlejs.baidu.com/humanStream

# 效果

从左到右:原图、背景虚化、背景替换、人型遮罩

<img width="800" src="https://user-images.githubusercontent.com/10822846/126873788-1e2d4984-274f-45be-8716-2a87ddda8c75.png"/>
<img width="800" src="https://user-images.githubusercontent.com/10822846/126873838-e5b68c9b-279f-4cb4-ae90-6aaaecd06aa4.png"/>


# 视频会议
<p>
<img width="400" src="https://user-images.githubusercontent.com/10822846/126872499-c3fd680e-a01b-4daa-b0cb-acd3290862bd.gif"/>
<img width="400" src="https://user-images.githubusercontent.com/10822846/126872930-4f4c5c5d-5c51-44fe-b2d6-3f83c4e124bc.png"/>
</p>

Loading