Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

阿里云盘通过 rsync 传输失败 #5512

Closed
4 tasks done
tigertall opened this issue Nov 12, 2023 · 5 comments
Closed
4 tasks done

阿里云盘通过 rsync 传输失败 #5512

tigertall opened this issue Nov 12, 2023 · 5 comments
Labels
bug Something isn't working invalid This doesn't seem right

Comments

@tigertall
Copy link

tigertall commented Nov 12, 2023 •
edited
Loading

Please make sure of the following things

  • I have read the documentation.
    我已经阅读了文档。

  • I'm sure there are no duplicate issues or discussions.
    我确定没有重复的issue或讨论。

  • I'm sure it's due to AList and not something else(such as Network ,Dependencies or Operational).
    我确定是AList的问题，而不是其他原因（例如网络，依赖或操作）。

  • I'm sure this issue is not fixed in the latest version.
    我确定这个问题在最新版本中没有被修复。

AList Version / AList 版本

docker compose version 3.3
Built At: 2023-09-22 14:57:18 +0000
Go Version: go1.20.8 linux/amd64
Author: Xhofe i@nn.ci
Commit ID: a64dd48
Version: v3.28.0
WebVersion: 3.28.0

Driver used / 使用的存储驱动

阿里云盘

Describe the bug / 问题描述

与 #5403 现象一致，区别在于我是使用 rsync + davfs2 做同步。
操作系统，debian 12，davfs2 挂载参数和配置为默认。
mount: sudo mount -t davfs http://127.0.0.1:5244/dav/alidrive /mnt/alidrive
rsync 同步命令为 ：sudo rsync -avP --size-only --delete --inplace --append --append-verify /opt/hdd1/nextcloud/data/tiger/files/tmp/ /mnt/alidrive/rasp/tiger/test/ （不带 inplace 效果一样）
另外尝试了 rsync 加 timeout 参数 也无效。

一些需要说明的情况：
1、文件大小14G
2、带宽网络 300Mb，上行估计为 30Mb。
3、rsync 完成较快，然后观察 alist 日志持续失败。
4、使用 alist 网页端，操作上传同一个文件到阿里云盘另外一个文件夹，能够成功。

Reproduction / 复现链接

可以稳定复现。

Config / 配置

阿里云盘配置：
缓存过期时间:30分钟
web代理:关闭
webdav策略：302重定向

Logs / 日志

�[36mINFO�[0m[2023-11-12 13:18:33] [ali_open] toekn exchange: .....->.....
[GIN] 2023/11/12 - 13:18:33 | 207 | 421.492388ms | 172.18.0.1 | PROPFIND "/dav/alidrive/"
[GIN] 2023/11/12 - 13:18:33 | 207 | 94.34517ms | 172.18.0.1 | PROPFIND "/dav/alidrive/rasp/"
[GIN] 2023/11/12 - 13:18:33 | 207 | 247.289613ms | 172.18.0.1 | PROPFIND "/dav/alidrive/rasp/tiger/"
[GIN] 2023/11/12 - 13:18:33 | 207 | 228.420328ms | 172.18.0.1 | PROPFIND "/dav/alidrive/rasp/tiger/test/"
[GIN] 2023/11/12 - 13:18:40 | 207 | 68.24298ms | 172.18.0.1 | PROPFIND "/dav/alidrive/rasp/tiger/test/"
[GIN] 2023/11/12 - 13:18:40 | 200 | 3.32835ms | 172.18.0.1 | LOCK "/dav/alidrive/rasp/tiger/test/apple7-XXX.zip"
[GIN] 2023/11/12 - 13:19:57 | 207 | 70.94799ms | 172.18.0.1 | PROPFIND "/dav/alidrive/rasp/tiger/test/"
[GIN] 2023/11/12 - 13:19:57 | 207 | 295.07µs | 172.18.0.1 | PROPFIND "/dav/alidrive/rasp/tiger/"
�[31mERRO�[0m[2023-11-12 13:21:07] failed put /alidrive/rasp/tiger/test: All attempts fail:
#1: Put "https://cn-beijing-data.aliyundrive.net/JN5Oo0fL%2F744496%2F6550d1089eccb46034d54429b9e99c175c3c9a1f%2F6550d108c82ab6f40f524c71984f886e9275f1cc?partNumber=7&security-token=CAIS%2BgF1q6Ft5B2yfSjIr5DGAcDCgOhgz4i6aXDlhTg6bvxYqq6cmzz2IHFPeHJrBeAYt%2FoxmW1X5vwSlq5rR4QAXlDfNVXECCuDqFHPWZHInuDox55m4cTXNAr%2BIhr%2F29CoEIedZdjBe%2FCrRknZnytou9XTfimjWFrXWv%2Fgy%2BQQDLItUxK%2FcCBNCfpPOwJms7V6D3bKMuu3OROY6Qi5TmgQ41Uh1jgjtPzkkpfFtkGF1GeXkLFF%2B97DRbG%2FdNRpMZtFVNO44fd7bKKp0lQLukMWr%2Fwq3PIdp2ma447NWQlLnzyCMvvJ9OVDFyN0aKEnH7J%2Bq%2FzxhTPrMnpkSlacGoABGUW2xqKMcP%2B9yfMoBbudVZSVZM6HsPrtKjSkEMNvlHZJR9SQ7HnEkBXW8hgzg%2BHJx%2FQgr0OtCQ1rhlpr6MY3i0mqfYBoC64nQGkMcA%2BmodVzhzdYoF57GvPxsh%2Fm5Yb1i1iVtQE53EMNXO8hWUDdq5JP2Zks2oY7h%2BPKr2lcckkgAA%3D%3D&uploadId=8D8B16D5EEDF41A68E34C57FA9503014&x-oss-access-key-id=STS.NSsJzvm7AxJQkVTa8obstEk7y&x-oss-expires=1699798808&x-oss-signature=3uAOXbOYHAoxwP0OejGIkrugD007ZkDUqs3%2B81wQVVY%3D&x-oss-signature-version=OSS2": context canceled
#2: Put "https://cn-beijing-data.aliyundrive.net/JN5Oo0fL%2F744496%2F6550d1089eccb46034d54429b9e99c175c3c9a1f%2F6550d108c82ab6f40f524c71984f886e9275f1cc?partNumber=7&security-token=CAIS%2BgF1q6Ft5B2yfSjIr5DGAcDCgOhgz4i6aXDlhTg6bvxYqq6cmzz2IHFPeHJrBeAYt%2FoxmW1X5vwSlq5rR4QAXlDfNVXECCuDqFHPWZHInuDox55m4cTXNAr%2BIhr%2F29CoEIedZdjBe%2FCrRknZnytou9XTfimjWFrXWv%2Fgy%2BQQDLItUxK%2FcCBNCfpPOwJms7V6D3bKMuu3OROY6Qi5TmgQ41Uh1jgjtPzkkpfFtkGF1GeXkLFF%2B97DRbG%2FdNRpMZtFVNO44fd7bKKp0lQLukMWr%2Fwq3PIdp2ma447NWQlLnzyCMvvJ9OVDFyN0aKEnH7J%2Bq%2FzxhTPrMnpkSlacGoABGUW2xqKMcP%2B9yfMoBbudVZSVZM6HsPrtKjSkEMNvlHZJR9SQ7HnEkBXW8hgzg%2BHJx%2FQgr0OtCQ1rhlpr6MY3i0mqfYBoC64nQGkMcA%2BmodVzhzdYoF57GvPxsh%2Fm5Yb1i1iVtQE53EMNXO8hWUDdq5JP2Zks2oY7h%2BPKr2lcckkgAA%3D%3D&uploadId=8D8B16D5EEDF41A68E34C57FA9503014&x-oss-access-key-id=STS.NSsJzvm7AxJQkVTa8obstEk7y&x-oss-expires=1699798808&x-oss-signature=3uAOXbOYHAoxwP0OejGIkrugD007ZkDUqs3%2B81wQVVY%3D&x-oss-signature-version=OSS2": context canceled
#3: Put "https://cn-beijing-data.aliyundrive.net/JN5Oo0fL%2F744496%2F6550d1089eccb46034d54429b9e99c175c3c9a1f%2F6550d108c82ab6f40f524c71984f886e9275f1cc?partNumber=7&security-token=CAIS%2BgF1q6Ft5B2yfSjIr5DGAcDCgOhgz4i6aXDlhTg6bvxYqq6cmzz2IHFPeHJrBeAYt%2FoxmW1X5vwSlq5rR4QAXlDfNVXECCuDqFHPWZHInuDox55m4cTXNAr%2BIhr%2F29CoEIedZdjBe%2FCrRknZnytou9XTfimjWFrXWv%2Fgy%2BQQDLItUxK%2FcCBNCfpPOwJms7V6D3bKMuu3OROY6Qi5TmgQ41Uh1jgjtPzkkpfFtkGF1GeXkLFF%2B97DRbG%2FdNRpMZtFVNO44fd7bKKp0lQLukMWr%2Fwq3PIdp2ma447NWQlLnzyCMvvJ9OVDFyN0aKEnH7J%2Bq%2FzxhTPrMnpkSlacGoABGUW2xqKMcP%2B9yfMoBbudVZSVZM6HsPrtKjSkEMNvlHZJR9SQ7HnEkBXW8hgzg%2BHJx%2FQgr0OtCQ1rhlpr6MY3i0mqfYBoC64nQGkMcA%2BmodVzhzdYoF57GvPxsh%2Fm5Yb1i1iVtQE53EMNXO8hWUDdq5JP2Zks2oY7h%2BPKr2lcckkgAA%3D%3D&uploadId=8D8B16D5EEDF41A68E34C57FA9503014&x-oss-access-key-id=STS.NSsJzvm7AxJQkVTa8obstEk7y&x-oss-expires=1699798808&x-oss-signature=3uAOXbOYHAoxwP0OejGIkrugD007ZkDUqs3%2B81wQVVY%3D&x-oss-signature-version=OSS2": context canceled
github.com/alist-org/alist/v3/internal/op.Put
/app/internal/op/fs.go:578
github.com/alist-org/alist/v3/internal/fs.putDirectly
/app/internal/fs/put.go:54
github.com/alist-org/alist/v3/internal/fs.PutDirectly
/app/internal/fs/fs.go:96
github.com/alist-org/alist/v3/server/webdav.(*Handler).handlePut
/app/server/webdav/webdav.go:341
github.com/alist-org/alist/v3/server/webdav.(*Handler).ServeHTTP
/app/server/webdav/webdav.go:65
github.com/alist-org/alist/v3/server.ServeWebDAV
/app/server/webdav.go:46
github.com/gin-gonic/gin.(*Context).Next
/root/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/context.go:174
github.com/alist-org/alist/v3/server.WebDAVAuth
/app/server/webdav.go:113
github.com/gin-gonic/gin.(*Context).Next
/root/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/context.go:174
github.com/alist-org/alist/v3/server/middlewares.StoragesLoaded
/app/server/middlewares/check.go:14
github.com/gin-gonic/gin.(*Context).Next
/root/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/context.go:174
github.com/gin-gonic/gin.CustomRecoveryWithWriter.func1
/root/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/recovery.go:102
github.com/gin-gonic/gin.(*Context).Next
/root/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/context.go:174
github.com/gin-gonic/gin.LoggerWithConfig.func1
/root/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/logger.go:240
github.com/gin-gonic/gin.(*Context).Next
/root/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/context.go:174
github.com/gin-gonic/gin.(*Engine).handleHTTPRequest
/root/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/gin.go:620
github.com/gin-gonic/gin.(*Engine).ServeHTTP
/root/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/gin.go:576
net/http.serverHandler.ServeHTTP
/usr/lib/go/src/net/http/server.go:2936
net/http.(*conn).serve
/usr/lib/go/src/net/http/server.go:1995
runtime.goexit
/usr/lib/go/src/runtime/asm_amd64.s:1598
�[31mERRO�[0m[2023-11-12 13:21:07] PUT /dav/alidrive/rasp/tiger/test/apple7-XXX.zip All attempts fail:
#1: Put "https://cn-beijing-data.aliyundrive.net/JN5Oo0fL%2F744496%2F6550d1089eccb46034d54429b9e99c175c3c9a1f%2F6550d108c82ab6f40f524c71984f886e9275f1cc?partNumber=7&security-token=CAIS%2BgF1q6Ft5B2yfSjIr5DGAcDCgOhgz4i6aXDlhTg6bvxYqq6cmzz2IHFPeHJrBeAYt%2FoxmW1X5vwSlq5rR4QAXlDfNVXECCuDqFHPWZHInuDox55m4cTXNAr%2BIhr%2F29CoEIedZdjBe%2FCrRknZnytou9XTfimjWFrXWv%2Fgy%2BQQDLItUxK%2FcCBNCfpPOwJms7V6D3bKMuu3OROY6Qi5TmgQ41Uh1jgjtPzkkpfFtkGF1GeXkLFF%2B97DRbG%2FdNRpMZtFVNO44fd7bKKp0lQLukMWr%2Fwq3PIdp2ma447NWQlLnzyCMvvJ9OVDFyN0aKEnH7J%2Bq%2FzxhTPrMnpkSlacGoABGUW2xqKMcP%2B9yfMoBbudVZSVZM6HsPrtKjSkEMNvlHZJR9SQ7HnEkBXW8hgzg%2BHJx%2FQgr0OtCQ1rhlpr6MY3i0mqfYBoC64nQGkMcA%2BmodVzhzdYoF57GvPxsh%2Fm5Yb1i1iVtQE53EMNXO8hWUDdq5JP2Zks2oY7h%2BPKr2lcckkgAA%3D%3D&uploadId=8D8B16D5EEDF41A68E34C57FA9503014&x-oss-access-key-id=STS.NSsJzvm7AxJQkVTa8obstEk7y&x-oss-expires=1699798808&x-oss-signature=3uAOXbOYHAoxwP0OejGIkrugD007ZkDUqs3%2B81wQVVY%3D&x-oss-signature-version=OSS2": context canceled
#2: Put "https://cn-beijing-data.aliyundrive.net/JN5Oo0fL%2F744496%2F6550d1089eccb46034d54429b9e99c175c3c9a1f%2F6550d108c82ab6f40f524c71984f886e9275f1cc?partNumber=7&security-token=CAIS%2BgF1q6Ft5B2yfSjIr5DGAcDCgOhgz4i6aXDlhTg6bvxYqq6cmzz2IHFPeHJrBeAYt%2FoxmW1X5vwSlq5rR4QAXlDfNVXECCuDqFHPWZHInuDox55m4cTXNAr%2BIhr%2F29CoEIedZdjBe%2FCrRknZnytou9XTfimjWFrXWv%2Fgy%2BQQDLItUxK%2FcCBNCfpPOwJms7V6D3bKMuu3OROY6Qi5TmgQ41Uh1jgjtPzkkpfFtkGF1GeXkLFF%2B97DRbG%2FdNRpMZtFVNO44fd7bKKp0lQLukMWr%2Fwq3PIdp2ma447NWQlLnzyCMvvJ9OVDFyN0aKEnH7J%2Bq%2FzxhTPrMnpkSlacGoABGUW2xqKMcP%2B9yfMoBbudVZSVZM6HsPrtKjSkEMNvlHZJR9SQ7HnEkBXW8hgzg%2BHJx%2FQgr0OtCQ1rhlpr6MY3i0mqfYBoC64nQGkMcA%2BmodVzhzdYoF57GvPxsh%2Fm5Yb1i1iVtQE53EMNXO8hWUDdq5JP2Zks2oY7h%2BPKr2lcckkgAA%3D%3D&uploadId=8D8B16D5EEDF41A68E34C57FA9503014&x-oss-access-key-id=STS.NSsJzvm7AxJQkVTa8obstEk7y&x-oss-expires=1699798808&x-oss-signature=3uAOXbOYHAoxwP0OejGIkrugD007ZkDUqs3%2B81wQVVY%3D&x-oss-signature-version=OSS2": context canceled
#3: Put "https://cn-beijing-data.aliyundrive.net/JN5Oo0fL%2F744496%2F6550d1089eccb46034d54429b9e99c175c3c9a1f%2F6550d108c82ab6f40f524c71984f886e9275f1cc?partNumber=7&security-token=CAIS%2BgF1q6Ft5B2yfSjIr5DGAcDCgOhgz4i6aXDlhTg6bvxYqq6cmzz2IHFPeHJrBeAYt%2FoxmW1X5vwSlq5rR4QAXlDfNVXECCuDqFHPWZHInuDox55m4cTXNAr%2BIhr%2F29CoEIedZdjBe%2FCrRknZnytou9XTfimjWFrXWv%2Fgy%2BQQDLItUxK%2FcCBNCfpPOwJms7V6D3bKMuu3OROY6Qi5TmgQ41Uh1jgjtPzkkpfFtkGF1GeXkLFF%2B97DRbG%2FdNRpMZtFVNO44fd7bKKp0lQLukMWr%2Fwq3PIdp2ma447NWQlLnzyCMvvJ9OVDFyN0aKEnH7J%2Bq%2FzxhTPrMnpkSlacGoABGUW2xqKMcP%2B9yfMoBbudVZSVZM6HsPrtKjSkEMNvlHZJR9SQ7HnEkBXW8hgzg%2BHJx%2FQgr0OtCQ1rhlpr6MY3i0mqfYBoC64nQGkMcA%2BmodVzhzdYoF57GvPxsh%2Fm5Yb1i1iVtQE53EMNXO8hWUDdq5JP2Zks2oY7h%2BPKr2lcckkgAA%3D%3D&uploadId=8D8B16D5EEDF41A68E34C57FA9503014&x-oss-access-key-id=STS.NSsJzvm7AxJQkVTa8obstEk7y&x-oss-expires=1699798808&x-oss-signature=3uAOXbOYHAoxwP0OejGIkrugD007ZkDUqs3%2B81wQVVY%3D&x-oss-signature-version=OSS2": context canceled
@tigertall tigertall added the bug Something isn't working label Nov 12, 2023
@welcome Welcome
Copy link

welcome bot commented Nov 12, 2023

Thanks for opening your first issue here! Be sure to follow the issue template!

@xhofe
Copy link
Collaborator

xhofe commented Nov 13, 2023

这依然为客户端超时取消了上传。请使用支持设置超时时间的客户端并设置超时时间。rsync并不是客户端 davfs2才是

@xhofe xhofe added the invalid This doesn't seem right label Nov 13, 2023
@github-actions GitHub Actions
Copy link

Hello @tigertall, your issue is invalid and will be closed.
你好 @tigertall，你的issue无效，将被关闭。

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Nov 13, 2023
@tigertall
Copy link
Author

感谢提示，调整 davfs2 配置参数 read_timeout 从默认的30 (seconds) 为7200之后，确实可以上传成功。
留言供其他需要的同学参考。

@Hhh3S
Copy link

Hhh3S commented Nov 29, 2023

群晖cloud sync不支持设置超时时间，囧

@tigertall tigertall mentioned this issue Mar 6, 2024
Closed
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working invalid This doesn't seem right
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@tigertall @Hhh3S @xhofe