Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fix] fix pytorch-paddle mapping index generate #6463

Merged
merged 20 commits into from
Jan 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions ci_scripts/gendoc.sh
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,10 @@ fi

if [ -f ${FLUIDDOCDIR}/ci_scripts/hooks/pre-doc-compile.sh ] ; then
${FLUIDDOCDIR}/ci_scripts/hooks/pre-doc-compile.sh
if [ $? -ne 0 ]; then
echo "pre-doc-compile.sh failed."
exit 1
fi
fi

thread=2
Expand Down
7 changes: 6 additions & 1 deletion ci_scripts/hooks/pre-doc-compile.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
SCRIPT_DIR="$( cd "$( dirname "$0" )" && pwd )"

FLUIDDOCDIR=${FLUIDDOCDIR:=/FluidDoc}
DOCROOT=${FLUIDDOCDIR}/docs/
DOCROOT=${FLUIDDOCDIR}/docs


## 1 merge the pytorch to paddle api map tables
Expand Down Expand Up @@ -44,3 +44,8 @@ done
APIMAPPING_ROOT=${DOCROOT}/guides/model_convert/convert_from_pytorch

python ${APIMAPPING_ROOT}/apply_reference_from_api_difference.py

if [ $? -ne 0 ]; then
echo "Error: API mapping generate script failed, please check changes in ${APIMAPPING_ROOT}"
exit 1
fi
1 change: 1 addition & 0 deletions docs/guides/model_convert/convert_from_pytorch/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
api_mappings.json
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
## [ 仅参数名不一致 ]torch.tensor_split
### [torch.tensor_split](https://pytorch.org/docs/stable/generated/torch.Tensor.tensor_split.html)
## [ 仅参数名不一致 ]torch.Tensor.tensor_split
### [torch.Tensor.tensor_split](https://pytorch.org/docs/stable/generated/torch.Tensor.tensor_split.html)

```python
torch.tensor_split(indices_or_sections, dim=0)
torch.Tensor.tensor_split(indices_or_sections, dim=0)
```

### [paddle.tensor_split](https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/api/paddle/Tensor_cn.html#tensor_split-num_or_indices-axis-0-name-none)
### [paddle.Tensor.tensor_split](https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/api/paddle/Tensor_cn.html#tensor_split-num_or_indices-axis-0-name-none)

```python
paddle.tensor_split(num_or_indices, axis=0, name=None)
paddle.Tensor.tensor_split(num_or_indices, axis=0, name=None)
```

其中 Paddle 相比 PyTorch 仅参数名不一致,具体如下:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
## [ 仅参数名不一致 ]torch.linalg.matrix_exp
### [torch.linalg.matrix_power](https://pytorch.org/docs/stable/generated/torch.linalg.matrix_exp.html#torch.linalg.matrix_exp)
### [torch.linalg.matrix_exp](https://pytorch.org/docs/stable/generated/torch.linalg.matrix_exp.html#torch.linalg.matrix_exp)

```python
torch.linalg.matrix_exp(A)
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
## [ 参数完全一致 ] torch.nn.AdaptiveAvgPool1d

### [torch.nn.AdaptiveAvgPool1d](https://pytorch.org/docs/stable/generated/torch.nn.AdaptiveAvgPool1d.html)

```python
torch.nn.AdaptiveAvgPool1d(output_size)
```

### [paddle.nn.AdaptiveAvgPool1D](https://www.paddlepaddle.org.cn/documentation/docs/zh/api/paddle/nn/AdaptiveAvgPool1D_cn.html#adaptiveavgpool1d)

```python
paddle.nn.AdaptiveAvgPool1D(output_size, name=None)
```

两者功能一致,参数完全一致。
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
## [ 仅 paddle 参数更多 ] torch.nn.AdaptiveAvgPool2d

### [torch.nn.AdaptiveAvgPool2d](https://pytorch.org/docs/stable/generated/torch.nn.AdaptiveAvgPool2d.html)

```python
torch.nn.AdaptiveAvgPool2d(output_size)
```

### [paddle.nn.AdaptiveAvgPool2D](https://www.paddlepaddle.org.cn/documentation/docs/zh/api/paddle/nn/AdaptiveAvgPool2D_cn.html#adaptiveavgpool2d)

```python
paddle.nn.AdaptiveAvgPool2D(output_size, data_format='NCHW', name=None)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个不是paddle参数更多?

```

其中 Paddle 相比 PyTorch 支持更多其他参数,具体如下:
### 参数映射
| PyTorch | PaddlePaddle | 备注 |
| ------------- | ------------ | ------------------------------------------------------ |
| output_size | output_size | 表示输出 Tensor 的 size 。 |
| - | data_format | 表示输入 Tensor 的数据格式, PyTorch 无此参数, Paddle 保持默认即可。 |
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
## [ 仅 paddle 参数更多 ] torch.nn.AdaptiveAvgPool3d

### [torch.nn.AdaptiveAvgPool3d](https://pytorch.org/docs/stable/generated/torch.nn.AdaptiveAvgPool3d.html)

```python
torch.nn.AdaptiveAvgPool3d(output_size)
```

### [paddle.nn.AdaptiveAvgPool3D](https://www.paddlepaddle.org.cn/documentation/docs/zh/api/paddle/nn/AdaptiveAvgPool3D_cn.html#adaptiveavgpool3d)

```python
paddle.nn.AdaptiveAvgPool3D(output_size, data_format='NCDHW', name=None)
```

其中 Paddle 相比 PyTorch 支持更多其他参数,具体如下:
### 参数映射
| PyTorch | PaddlePaddle | 备注 |
| ------------- | ------------ | ------------------------------------------------------ |
| output_size | output_size | 表示输出 Tensor 的 size 。 |
| - | data_format | 表示输入 Tensor 的数据格式, PyTorch 无此参数, Paddle 保持默认即可。 |
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
## [ 无参数 ] torch.nn.LogSigmoid

### [torch.nn.LogSigmoid](https://pytorch.org/docs/stable/generated/torch.nn.LogSigmoid.html)

```python
torch.nn.LogSigmoid(*args, **kwargs)
```

### [paddle.nn.LogSigmoid](https://www.paddlepaddle.org.cn/documentation/docs/zh/api/paddle/nn/LogSigmoid_cn.html#logsigmoid)

```python
paddle.nn.LogSigmoid(name=None)
```

两者功能一致,均无参数。
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
## [ 无参数 ] torch.nn.Sigmoid

### [torch.nn.Sigmoid](https://pytorch.org/docs/stable/generated/torch.nn.Sigmoid.html)

```python
torch.nn.Sigmoid(*args, **kwargs)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个是不是torch可以输入参数,paddle不能输入。torch参数更多?

```

### [paddle.nn.Sigmoid](https://www.paddlepaddle.org.cn/documentation/docs/zh/api/paddle/nn/Sigmoid_cn.html#sigmoid)

```python
paddle.nn.Sigmoid(name=None)
```

两者功能一致,均无参数。
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
## [ 参数完全一致 ] torch.nn.Softplus

### [torch.nn.Softplus](https://pytorch.org/docs/stable/generated/torch.nn.Softplus.html)

```python
torch.nn.Softplus(beta=1, threshold=20)
```

### [paddle.nn.Softplus](https://www.paddlepaddle.org.cn/documentation/docs/zh/api/paddle/nn/Softplus_cn.html)

```python
paddle.nn.Softplus(beta=1, threshold=20, name=None)
```

两者功能一致,参数完全一致。
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
## [ 无参数 ] torch.nn.Softsign

### [torch.nn.Softsign](https://pytorch.org/docs/stable/generated/torch.nn.Softsign.html)

```python
torch.nn.Softsign(*args, **kwargs)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个是不是torch可以输入参数,paddle不能输入。torch参数更多?

```

### [paddle.nn.Softsign](https://www.paddlepaddle.org.cn/documentation/docs/zh/api/paddle/nn/Softsign_cn.html)

```python
paddle.nn.Softsign(name=None)
```

两者功能一致,均无参数。
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
## [ 无参数 ] torch.nn.Tanh

### [torch.nn.Tanh](https://pytorch.org/docs/stable/generated/torch.nn.Tanh.html)

```python
torch.nn.Tanh(*args, **kwargs)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个是不是torch可以输入参数,paddle不能输入。torch参数更多?

```

### [paddle.nn.Tanh](https://www.paddlepaddle.org.cn/documentation/docs/zh/api/paddle/nn/Tanh_cn.html)

```python
paddle.nn.Tanh(name=None)
```

两者功能一致,无参数。
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
## [ 无参数 ] torch.nn.Tanhshrink

### [torch.nn.Tanhshrink](https://pytorch.org/docs/stable/generated/torch.nn.Tanhshrink.html)

```python
torch.nn.Tanhshrink(*args, **kwargs)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个是不是torch可以输入参数,paddle不能输入。torch参数更多?

```

### [paddle.nn.Tanhshrink](https://www.paddlepaddle.org.cn/documentation/docs/zh/api/paddle/nn/Tanhshrink_cn.html)

```python
paddle.nn.Tanhshrink(name=None)
```

两者功能一致,均无参数。
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
## [ 参数完全一致 ] torch.nn.TripletMarginWithDistanceLoss

### [torch.nn.TripletMarginWithDistanceLoss](https://pytorch.org/docs/stable/generated/torch.nn.TripletMarginWithDistanceLoss.html)

```python
torch.nn.TripletMarginWithDistanceLoss(*, distance_function=None, margin=1.0, swap=False, reduction='mean')
```

### [paddle.nn.TripletMarginWithDistanceLoss](https://www.paddlepaddle.org.cn/documentation/docs/zh/api/paddle/nn/TripletMarginWithDistanceLoss_cn.html#tripletmarginwithdistanceloss)

```python
paddle.nn.TripletMarginWithDistanceLoss(distance_function=None, margin: float = 1.0, swap: bool = False, reduction: str = 'mean', name: str = None)
```

两者功能一致,参数完全一致。
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
## [ 参数不一致 ]torch.Tensor.slice_scatter
## [ 参数不一致 ]torch.slice_scatter

### [torch.slice_scatter](https://pytorch.org/docs/stable/generated/torch.slice_scatter.html#torch.slice_scatter)

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import json
import os
import re
import sys
import typing


Expand Down Expand Up @@ -207,8 +208,10 @@ def apply_reference_to_row(line, metadata_dict, table_row_idx, line_idx):
output = "| " + " | ".join(content) + " |\n"
return output
else:
print(f"found manual-maintaining row at line [{line_idx}]: {line}")
return line
raise ValueError(
f"found manual-maintaining row at line [{line_idx}]: {line}"
)
# return line


def reference_mapping_item(index_path, metadata_dict):
Expand All @@ -218,6 +221,7 @@ def reference_mapping_item(index_path, metadata_dict):
with open(mapping_index_file, "r", encoding="utf-8") as f:
lines = f.readlines()

ret_code = 0
state = 0
# -1: error
# 0: wait for table header
Expand Down Expand Up @@ -309,12 +313,14 @@ def reference_mapping_item(index_path, metadata_dict):
print(e)
print(f"Error at line {i+1}: {line}")
output.append(line)
ret_code = 1

# state = 6
else:
raise Exception(
f"Unexpected State at {state} in processing file: {index_path}"
)
ret_code = 2

if state == 5 or state == 6:
state = 0
Expand All @@ -324,6 +330,9 @@ def reference_mapping_item(index_path, metadata_dict):
f"Unexpected End State at {state} in parsing file: {index_path}"
)

if ret_code != 0:
sys.exit(ret_code)

with open(mapping_index_file, "w", encoding="utf-8") as f:
f.writelines(output)

Expand Down
Loading