Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ISSUE #4979] Canal Connector supports bidirectional data synchronization #5011

Merged
merged 4 commits into from
Jul 1, 2024

Conversation

xwm1992
Copy link
Contributor

@xwm1992 xwm1992 commented Jun 28, 2024

Fixes #4979

Motivation

Explain the content here.
Explain why you want to make the changes and what problem you're trying to solve.

Modifications

Describe the modifications you've done.

Documentation

  • Does this pull request introduce a new feature? (yes / no)
  • If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)
  • If a feature is not applicable for documentation, explain why?
  • If a feature is not documented yet in this PR, please create a followup issue for adding the documentation

@xwm1992 xwm1992 force-pushed the eventmesh-function branch from 40e893f to 50a204e Compare June 28, 2024 09:25
Copy link

codecov bot commented Jun 28, 2024

Codecov Report

Attention: Patch coverage is 0% with 14 lines in your changes missing coverage. Please review.

Project coverage is 15.63%. Comparing base (1c22c16) to head (50a204e).
Report is 37 commits behind head on eventmesh-function.

Files Patch % Lines
.../config/connector/rdb/canal/CanalSourceConfig.java 0.00% 7 Missing ⚠️
...on/config/connector/rdb/canal/CanalSinkConfig.java 0.00% 5 Missing ⚠️
...tmesh/common/remote/response/FetchJobResponse.java 0.00% 1 Missing ⚠️
.../common/remote/response/FetchPositionResponse.java 0.00% 1 Missing ⚠️
Additional details and impacted files
@@                   Coverage Diff                    @@
##             eventmesh-function    #5011      +/-   ##
========================================================
- Coverage                 16.50%   15.63%   -0.88%     
+ Complexity                 1823     1778      -45     
========================================================
  Files                       910     1000      +90     
  Lines                     32420    33755    +1335     
  Branches                   2796     2912     +116     
========================================================
- Hits                       5350     5276      -74     
- Misses                    26551    27960    +1409     
  Partials                    519      519              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Member

@Pil0tXia Pil0tXia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Pil0tXia Pil0tXia added the ready for review PR is waiting for reviewer's approval or opinion (used as a strong reminder) label Jun 29, 2024
Copy link
Member

@pchengma pchengma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@xwm1992 xwm1992 merged commit ae621d4 into apache:eventmesh-function Jul 1, 2024
10 of 12 checks passed
xwm1992 added a commit that referenced this pull request Jul 2, 2024
* EventMesh function admin (#4851)

* own

* dependency

* finish registry

* EventMesh function admin (#4853)

* own

* dependency

* finish registry

* init

* Eventmesh function admin (#4854)

* own

* dependency

* finish registry

* init

* 0419

* 0419

* more discovery and move gRPC

* fix dependency

* EventMesh function connector runtime (#4858)

* [ISSUE #4812] Set up Admin Endpoints v2 (#4813)

* Remove redundant overloaded methods

* Simplify write() result param

* Add writeJson(); Add PUT; Add JavaDoc

* Rename EventHttpHandler to EventMeshHttpHandler

* Correct server thread name

* Clean up messy & non-hierarchical overloading

* No need to set headers manually any more

* Set up v1&v2 endpoints

* Set up v1&v2 response dto

* Introduce fastjson2

* Fix fastjson2 "level too large : 2048" error caused by IPAddress

* Correct @ConfigField naming

* Return properties format json key

* Add format option to query string

* Introduce Result

* Reduce duplicate builder code

* Fix all checkstyle warnings in eventmesh-runtime

* Add known dependency

* [ISSUE #4814] Migrate from fastjson 1.2.83 to fastjson2 (#4819)

* [Enhancement] Migrate from fastjson 1.2.83 to fastjson2 #4814

* fix_dependencies_problem

* fix_check

* [ISSUE #4551] modify the logic of time-consumption statistics (#4822)

* init connector runtime v2

* [ISSUE #4804] Fix SubStreamHandler exception loop by closeOnError (#4807)

* Handle exception loop by closeOnError

* Lombok optimization

* some format optimization

* Avoid closing multiple times

* Remove redundant set null

* Revert "Avoid closing multiple times"

This reverts commit 774397f.

* Use synchronized latch to keep senderOnComplete called once

* Use boolean to prevent latch called by somebody else

* Remove the unique callee/caller close() of onCompleted()

* [ISSUE #4838] Deprecate unused `eventMesh.connector.plugin.type` etc. properties (#4839)

* Remove all references of `eventMesh.connector.plugin.type`

* Deprecate `eventMesh.connector.plugin.type` and sort properties

* Remove misconfigured & not-used `registerIntervalInMills`, `fetchRegistryAddrIntervalInMills`

* Remove 'defibus' related un-used usages

* Supplement #4809 for `null != object`

* [ISSUE #4832] Downgrade stale bot to v8 to resolve state cache reserving error (#4833)

* Revert stale bot to v8 to resolve state cache reserving error

* Reduce operations-per-run to default value to ease pressure

* Unify yaml to yml

* [ISSUE #4820] Bug fix EventHandler not return json (#4821)

* bug fix

* bug fix

* bug fix

* update runtime v2

* update connector runtime

* update connector runtime

* update connector runtime

* update connector runtime

* update connector runtime

---------

Co-authored-by: Pil0tXia <xiatian@apache.org>
Co-authored-by: Zaki <91261012+cnzakii@users.noreply.github.com>
Co-authored-by: Karson <karsontao@hotmail.com>

* [ISSUE #4931]Add Registry Module for Discovery AdminServer

* [ISSUES #4933]Add Admin Module

* [ISSUE #4935] Add and Move the Pojo Used By Both Runtime and Admin to Common

* [ISSUE #4937]fix gradle dependecy and add runtime v2

* [ISSUES #4939]add canal connector

* fix missing apache header

* fix missing apache header

* fix missing apache header

* update gradle dependencies

* fix admin server ci check error

* fix admin server ci check error

* fix ci checkStyle error

* fix ci check error

* [ISSUE #4979]Canal Connector supports bidirectional data synchronization

* add bash files for admin & runtime-v2

* fix ack offset read & persist

* fix checkStyle error

* [ISSUE #4979] Canal Connector supports bidirectional data synchronization (#5011)

* [ISSUE #4979]Canal Connector supports bidirectional data synchronization

* add bash files for admin & runtime-v2

* fix ack offset read & persist

* fix checkStyle error

* fix http source connector test error

---------

Co-authored-by: sodaRyCN <35725024+sodaRyCN@users.noreply.github.com>
Co-authored-by: Pil0tXia <xiatian@apache.org>
Co-authored-by: Zaki <91261012+cnzakii@users.noreply.github.com>
Co-authored-by: Karson <karsontao@hotmail.com>
@xwm1992 xwm1992 added this to the 1.11.0 milestone Dec 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready for review PR is waiting for reviewer's approval or opinion (used as a strong reminder)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants