Feature(wxh): Add FedAMP algo and fix bugs. #25
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Collaborator
XinghaoWu
commented
Oct 25, 2023
XinghaoWu
commented
- Add FedAMP algo.
- Add the FedCAC paper link in README.md.
- Remove the default momentum in default_config.py to fix the bug when setting the optimizer to Adam.
2. Add FedCAC paper link in README.md. 3. Remove the default momentum in default_config.py to fix the bug when set optmizer to adam.
kxzxvbk
changed the title
kxzxvbk
reviewed
Oct 26, 2023
| class FedAMPClient(BaseClient): | ||
| """ | ||
| Overview: | ||
| This class is the base implementation of client in 'Bold but Cautious: Unlocking the Potential of Personalized |
There was a problem hiding this comment.
Correct this document, not FedCAC.
kxzxvbk
reviewed
Oct 26, 2023
|
|
||
| def __init__(self, args, client_id, train_dataset, test_dataset=None): | ||
| """ | ||
| Initializing train dataset, test dataset(for personalized settings). |
There was a problem hiding this comment.
Correct this document, the purpose is to get a copy of local model.
kxzxvbk
reviewed
Oct 26, 2023
| super(FedAMPClient, self).__init__(args, client_id, train_dataset, test_dataset) | ||
| self.client_u = copy.deepcopy(self.model) | ||
|
|
||
| def FedAMP_Loss_client(self): |
There was a problem hiding this comment.
You can use the get_model_difference function (defined in fling/utils/torch_utils.py ) for simplification.
kxzxvbk
reviewed
Oct 26, 2023
| from fling.utils.utils import weight_flatten | ||
|
|
||
| @CLIENT_REGISTRY.register('fedamp_client') | ||
| class FedAMPClient(BaseClient): |
There was a problem hiding this comment.
I'm wondering whether this client is identical to FedProxClient? What's the differences?
kxzxvbk
reviewed
Oct 26, 2023
| coef = torch.zeros(self.args.client.client_num) | ||
| for j, mw in enumerate(self.client_ws): | ||
| if i == j: continue | ||
| sub = weights[i] - weights[j] |
There was a problem hiding this comment.
Rewrite it using fling.utils.get_model_difference
kxzxvbk
reviewed
Oct 26, 2023
kxzxvbk
reviewed
Oct 26, 2023
| @@ -14,6 +15,13 @@ def client_sampling(client_ids: Iterable, sample_rate: float) -> List: | |||
| ) | |||
| return participated_clients | |||
|
|
|||
| def weight_flatten(model) -> torch.Tensor: | |||
There was a problem hiding this comment.
I suppose that this function can be removed.
|
Reformat the code before final merge. |
|
Add example configs for cifar100, mnist and tiny-imagenet. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.