Dev 3.1.0 #720
Merged
Dev 3.1.0 #720
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
create WrappedXX class to mimic old eggroll api Signed-off-by: sagewe <wbwmat@gmail.com>
- Introduced `random` and `typing` (Callable, Iterable, Any, Tuple) imports for advanced operations and type hinting. - Added comprehensive methods to WrappedRp class including `delete`, `count`, `get_all`, `put_all`, `take`, `first`, `destroy`, and various map-reduce functions. - Implemented serialization and deserialization support using UnrestrictedSerdes for robust data handling. - Extended map-reduce capabilities with custom partition operations, sampling, filtering, repartitioning, and binary operations on sorted partitions. - Included specialized merge functions for union, join, and subtract operations on keys. - Organized and optimized functions with private utility methods for serialization and iteration processing. - Ensured compatibility and efficiency with thoughtful error handling and condition checks. Signed-off-by: sagewe <wbwmat@gmail.com>
1. impl rollsite wrap 2. add monkey patch to patch package import use custom package loader 3. refactor some class Signed-off-by: sagewe <wbwmat@gmail.com>
since eggroll 2.x use protobuf version is 3.x while eggroll 3.x use protobuf version 4.x, we need a way to import packages generated with correct version. since we introduce custom package loader before, we can achieve goals by 1. regenerate proto files with protobuf 3.x to backport directory 2. insert custom package loader to the meta_path head to make sure import of 'eggroll.core.proto' can be hijacked Signed-off-by: sagewe <wbwmat@gmail.com>
use `EGGROLL_2X_BACKPORT` env to enable backport and python/eggroll/backport directory can safety delete if you don't need this feature. Signed-off-by: sagewe <wbwmat@gmail.com>
1. add some missing requirements to requirements.txt 2. add requirements-2.x.txt for backport situation since protobuf version is different Signed-off-by: sagewe <wbwmat@gmail.com>
Signed-off-by: sagewe <wbwmat@gmail.com>
Signed-off-by: sagewe <wbwmat@gmail.com>
Signed-off-by: sagewe <wbwmat@gmail.com>
Signed-off-by: sagewe <wbwmat@gmail.com>
Signed-off-by: sagewe <wbwmat@gmail.com>
Signed-off-by: sagewe <wbwmat@gmail.com>
Signed-off-by: sagewe <wbwmat@gmail.com>
Signed-off-by: sagewe <wbwmat@gmail.com>
Signed-off-by: v_wbxiongli <740332065@qq.com>
update get gpu count
…ckport # Conflicts: # requirements.txt
Signed-off-by: sagewe <wbwmat@gmail.com>
backport eggroll 2.x to support FATE-1.x
Signed-off-by: Xiongli <740332065@qq.com>
# Conflicts: # python/eggroll/__init__.py
update version 3.1.0
Signed-off-by: v_wbxiongli <740332065@qq.com>
merge Feature 3.1.0 gpu
Signed-off-by: sagewe <wbwmat@gmail.com>
fix: fix restricted serde (#708)
Signed-off-by: sagewe <wbwmat@gmail.com>
fix: add docs and whitelist (#708)
dylan-fan
approved these changes
Mar 1, 2024
dylan-fan
approved these changes
Mar 1, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Merge dev-3.1.0 to main
Changes:
2.bug fix