Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

是否要求运行RedisShake节点要能够同时访问源跟目的Redis的所有节点吗? #876

Open
linshenyuan00 opened this issue Oct 29, 2024 · 8 comments
Labels
type: question Further information is requested

Comments

@linshenyuan00
Copy link

问题描述(Issue Description)

两个Redis部署在不同的环境上,之间的内网IP不通,是否这种就无法同步?
(即跨云环境,网络需要特别打通吗?)
(我想将aws上的ElastiCache Redis OSS迁移到其他云上自建的Redis 集群上。)
目前RedisShake是运行在跟ElastiCache 同VPC下的一个云主机上面,而目的Redis集群是在另外的云的VPC上。

Please provide a brief description of the issue you encountered.

环境信息(Environment)

RedisShake 版本(RedisShake Version):v4.2.0
Redis 源端版本(Redis Source Version):7.0版本
Redis 目的端版本(Redis Destination Version):7.0版本
Redis 部署方式(standalone/cluster/sentinel):cluster

  • 是否在云服务商实例上部署(Deployed on Cloud Provider):是

日志信息(Logs)

2024-10-29 01:52:36 INF load config from file: /home/linyh1/shake.toml
2024-10-29 01:52:36 INF log_level: [info], log_file: [/home/linyh1/data/shake.log]
2024-10-29 01:52:36 INF changed work dir. dir=[/home/linyh1/data]
2024-10-29 01:52:36 INF GOMAXPROCS defaults to the value of runtime.NumCPU [1]
2024-10-29 01:52:36 INF not set pprof port
2024-10-29 01:52:36 INF address=myredis.unwzow.clustercfg.apne1.cache.amazonaws.com:6379, reply=acffbf2015baa82698f5c41a3ca2a88173e58408 172.31.0.207:6379@1122 slave 613711e7fac6e6713090d2c911b05c9c7b02d6fc 0 1730166755000 1 connected
20623f07bbfdde339314b85f0c97029414a8a233 172.31.15.83:6379@1122 master - 0 1730166755000 2 connected 0-5461
7dcf82e533b22a7a16c11ca6d3a6d0b5169df0bd 172.31.28.8:6379@1122 slave 4dc93fd30659b0588e356ae783ea2e8bd19940e9 0 1730166756281 0 connected
7b028448b5d75d638083d55f13c712e288469936 172.31.26.195:6379@1122 slave 20623f07bbfdde339314b85f0c97029414a8a233 0 1730166755273 2 connected
4dc93fd30659b0588e356ae783ea2e8bd19940e9 172.31.46.183:6379@1122 master - 0 1730166755000 0 connected 10923-16383
613711e7fac6e6713090d2c911b05c9c7b02d6fc 172.31.18.206:6379@1122 myself,master - 0 1730166755000 1 connected 5462-10922
2024-10-29 01:52:36 INF [reader_172.31.15.83_6379] scanStandaloneReader init finished. dbs=[[0]]
2024-10-29 01:52:36 INF [reader_172.31.46.183_6379] scanStandaloneReader init finished. dbs=[[0]]
2024-10-29 01:52:36 INF [reader_172.31.18.206_6379] scanStandaloneReader init finished. dbs=[[0]]
2024-10-29 01:52:36 INF create ScanClusterReader: myredis.unwzow.clustercfg.apne1.cache.amazonaws.com:6379
2024-10-29 01:52:36 INF address=43.254.156.170:6379, reply=e325bba477417d19767fde182c649bb9d7b3ccd9 192.168.0.85:6379@16379 slave 1f2450c3dff57ab30a1114e3ea6c83d3f795573e 0 1730166756908 3 connected
0d57deaa7533fe00f37196426a7f60502f8867bd 192.168.0.83:6379@16379 master - 0 1730166754897 2 connected 5461-10922
1f2450c3dff57ab30a1114e3ea6c83d3f795573e 192.168.0.84:6379@16379 master - 0 1730166753891 3 connected 10923-16383
301dec657b976c46f4eac78e607e18d41dcd0489 192.168.0.87:6379@16379 slave 0d57deaa7533fe00f37196426a7f60502f8867bd 0 1730166755000 2 connected
8d02a021c3ff175f5d993ac9c9d3503c274de5c4 192.168.0.86:6379@16379 slave da038ed86f1a6f3ba65a3d5ce47674b986d2248c 0 1730166756000 1 connected
da038ed86f1a6f3ba65a3d5ce47674b986d2248c 192.168.0.82:6379@16379 myself,master - 0 1730166754000 1 connected 0-5460
2024-10-29 01:52:37 ERR dial failed. address=[192.168.0.83:6379], tls=[false], err=[dial tcp 192.168.0.83:6379: i/o timeout]
RedisShake/internal/client/redis.go:49 -> NewRedisClient()
RedisShake/internal/writer/redis_standalone_writer.go:51 -> NewRedisStandaloneWriter()
RedisShake/internal/writer/redis_cluster_writer.go:40 -> (*RedisClusterWriter).loadClusterNodes()
RedisShake/internal/writer/redis_cluster_writer.go:23 -> NewRedisClusterWriter()
RedisShake/cmd/redis-shake/main.go:101 -> main()
runtime/proc.go:267 -> main()
runtime/asm_amd64.s:1650 -> goexit()

其他信息(Additional Information)

.

@linshenyuan00 linshenyuan00 added the type: question Further information is requested label Oct 29, 2024
@suxb201
Copy link
Member

suxb201 commented Oct 29, 2024

是的,要求运行 RedisShake 的节点要能够同时访问源跟目的 Redis 的所有节点

@linshenyuan00
Copy link
Author

是否可以直接配置目的Redis的公网端口,来规避这种问题呢?

@suxb201
Copy link
Member

suxb201 commented Oct 29, 2024

可以的

@linshenyuan00
Copy link
Author

请问应该怎么配置呢? 有具体的文档吗?

@suxb201
Copy link
Member

suxb201 commented Oct 29, 2024

@linshenyuan00
Copy link
Author

我看 https://tair-opensource.github.io/RedisShake/zh/writer/redis_writer.html
只有介绍 redis_writer 如果是集群,配置一个节点就行了,
这种场景是说address 配置所有对外映射的节点吗?
还是其他参数呢?

@suxb201
Copy link
Member

suxb201 commented Oct 29, 2024

@linshenyuan00 明白你意思了。
你是如何给目的端开公网的?如果目的端 Redis listen 的 ip 就是公网 ip,那没问题,直接配置上就能跑。
如果是通过代理开公网,你需要修改较多 RedisShake 的代码,支持手动指定目的端地址才行。

@linshenyuan00
Copy link
Author

好的 那我知道了 谢谢

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants