Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(framework) Add flwr log #3577

Open
wants to merge 98 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
98 commits
Select commit Hold shift + click to select a range
bffea29
Add log cli
chongshenng Jun 11, 2024
aa71929
Update
chongshenng Jun 11, 2024
9d4f666
Refactor
chongshenng Jun 11, 2024
f46cc30
Lint
chongshenng Jun 11, 2024
2110415
Fix isort
chongshenng Jun 11, 2024
a68dfcd
Restructure code
chongshenng Jun 11, 2024
90e2aa7
Fix untyped calls
chongshenng Jun 11, 2024
96b6ba4
Add types
chongshenng Jun 11, 2024
3fe5fdc
Add type
chongshenng Jun 11, 2024
8dd95ae
Fix imports
chongshenng Jun 11, 2024
6a57e95
Fix pylint error and warnings
chongshenng Jun 11, 2024
a4c64c2
Move try-except inside if-else
chongshenng Jun 11, 2024
7ee30a2
Black
chongshenng Jun 11, 2024
3a69f75
Remove comments
chongshenng Jun 11, 2024
f2031d9
Update docstring
chongshenng Jun 12, 2024
a301049
Add refresh period to input arg
chongshenng Jun 12, 2024
3d5bd50
Fix arg
chongshenng Jun 12, 2024
aa84fc7
Add SuperExec address to input arg
chongshenng Jun 12, 2024
f95fba6
Lint
chongshenng Jun 12, 2024
55a6872
Add config_utils
chongshenng Jun 12, 2024
65df4ac
Merge main
chongshenng Jun 12, 2024
567e7e5
Move functions
chongshenng Jun 13, 2024
034e298
Handle run_id not found
chongshenng Jun 13, 2024
fbabd61
Merge branch 'main' into add-flwr-log
chongshenng Jun 13, 2024
072674e
Merge main
chongshenng Jun 14, 2024
29d0dd2
Update get_flwr_dir
chongshenng Jun 14, 2024
5d1cf6f
Merge branch 'main' into add-flwr-log
chongshenng Jun 14, 2024
8cdffba
Lint
chongshenng Jun 14, 2024
fe63096
Fix bug
chongshenng Jun 14, 2024
d0a23c7
Add autocancel
chongshenng Jun 14, 2024
4448a23
Standardize log info
chongshenng Jun 14, 2024
4ef4290
Merge branch 'main' into add-flwr-log
chongshenng Jun 19, 2024
a8b1654
Merge branch 'main' into add-flwr-log
tanertopal Jul 12, 2024
07a3ebf
feat(framework) Capture `node_id`/`node_config` in `Context` via `Nod…
jafermarq Jul 12, 2024
596f859
feat(framework) Add `node-config` arg to SuperNode (#3782)
charlesbvll Jul 12, 2024
889eadf
feat(framework) Introduce new `client_fn` signature passing the `Cont…
jafermarq Jul 13, 2024
76244be
refactor(framework) Remove `partition_id` from `Context` (#3792)
jafermarq Jul 13, 2024
ca48ae1
ci(*:skip) Update client_fn args in e2e tests (#3775)
danielnugraha Jul 13, 2024
5790759
docs(framework) Update client_fn docstrings to new signature (#3793)
danieljanes Jul 13, 2024
e8bf5f8
refactor(framework) Update `flwr new` templates with new `client_fn` …
jafermarq Jul 13, 2024
942adfd
feat(framework) Add SuperExec `--executor-config` (#3720)
charlesbvll Jul 14, 2024
3b5963f
refactor(framework) Update `flwr new` templates with new `server_fn` …
jafermarq Jul 14, 2024
6aff69d
fix(framework:skip) Remove dependency installation from SuperExec (#3…
charlesbvll Jul 14, 2024
ba20db4
refactor(framework) Rename `--config` to `--run-config` (#3798)
danieljanes Jul 14, 2024
b0de25d
docs(framework) Document public/private API approach (#3562)
danieljanes Jul 14, 2024
0e32d83
fix(framework:skip) Use correct arguments (#3799)
charlesbvll Jul 14, 2024
27ee0db
feat(framework) Add simulation engine `SuperExec` plugin (#3589)
jafermarq Jul 15, 2024
848734d
refactor(framework) Replace `run_id` with `Run` in simulation (#3802)
jafermarq Jul 15, 2024
ce60e52
refactor(framework) Register `Context` early in Simulation Engine (#3…
jafermarq Jul 15, 2024
dd37449
feat(framework) Use federations config in `flwr run` (#3800)
charlesbvll Jul 15, 2024
0f7c0f7
refactor(framework) Refactor `ClientApp` loading to use explicit argu…
jafermarq Jul 15, 2024
17049ca
feat(framework) Add federation argument to `flwr run` (#3807)
charlesbvll Jul 15, 2024
9988592
refactor(framework) Improve app loading in simulation engine (#3806)
jafermarq Jul 15, 2024
1e14dc6
refactor(framework) Switch to `tool.flwr` instead of `flower` in `pyp…
charlesbvll Jul 15, 2024
8f5b4e5
feat(framework) Add secure channel support for SuperExec (#3808)
charlesbvll Jul 15, 2024
9f78948
Set refresh period as constant
chongshenng Jul 16, 2024
27f314b
Merge branch 'main' into add-flwr-log
chongshenng Jul 16, 2024
f846ce0
Merge branch 'main' into add-flwr-log
tanertopal Jul 18, 2024
19fcc3a
Merge branch 'main' into add-flwr-log
chongshenng Jul 18, 2024
603cee6
Use directory and federation_name
chongshenng Jul 18, 2024
fbdb35c
Add pyproject.toml validation
chongshenng Jul 18, 2024
51d1700
Add parsing server_address from pyproject.toml
chongshenng Jul 18, 2024
471dec8
Remove unused imports
chongshenng Jul 18, 2024
3a4d40f
Merge branch 'main' into add-flwr-log
chongshenng Jul 19, 2024
3bd943d
Run isort
chongshenng Jul 19, 2024
8abe6c3
Remove default value for run_id
chongshenng Jul 19, 2024
183687b
Merge branch 'main' into add-flwr-log
chongshenng Jul 22, 2024
77a93e0
Merge branch 'main' into add-flwr-log
tanertopal Jul 22, 2024
940524a
Merge branch 'main' into add-flwr-log
chongshenng Jul 23, 2024
d278547
Merge branch 'main' into add-flwr-log
charlesbvll Jul 23, 2024
1f28cf6
Merge branch 'main' into add-flwr-log
chongshenng Jul 23, 2024
463f752
Set log level to debug
chongshenng Jul 23, 2024
3b04c29
fix(framework:skip) Use full name for HuggingFace template (#3883)
charlesbvll Jul 23, 2024
ae57fdb
fix(framework:skip) Fix `SimulationEngine` log (#3888)
chongshenng Jul 23, 2024
3c37de2
refactor(framework:skip) Improve SuperExec docs (#3889)
charlesbvll Jul 23, 2024
06417b9
fix(framework:skip) Unify default `client_resources` and deprecate se…
jafermarq Jul 23, 2024
7a89b4b
docs(datasets) Rewrite Flower Datasets quickstart tutorial as a noteb…
adam-narozniak Jul 23, 2024
e468fd1
fix(framework:skip) Show `DeploymentEngine` logs on SuperExec (#3891)
chongshenng Jul 24, 2024
be636b4
feat(framework:skip) Improve error handling with simulation executor …
charlesbvll Jul 24, 2024
e1bdc7e
Merge main
chongshenng Jul 25, 2024
da9518b
Update flwr log stream argument
chongshenng Jul 25, 2024
66fde74
Merge main
chongshenng Jul 25, 2024
af750b7
Update PR
chongshenng Aug 12, 2024
6d7c808
Remove abbreviations from log CLI
chongshenng Aug 12, 2024
e8d184c
Merge branch 'main' into add-flwr-log
chongshenng Aug 13, 2024
a41e4db
Add sys.exit when no address is given
chongshenng Aug 14, 2024
157aaf0
Merge branch 'main' into add-flwr-log
chongshenng Sep 9, 2024
f6a105a
Merge branch 'main' into add-flwr-log
tanertopal Sep 9, 2024
84df08b
Merge branch 'main' into add-flwr-log
chongshenng Sep 13, 2024
dd870d5
Ruff
chongshenng Sep 13, 2024
2bf93c0
Merge branch 'main' into add-flwr-log
chongshenng Sep 19, 2024
8ed01f7
Merge branch 'main' into add-flwr-log
chongshenng Sep 19, 2024
d6c042a
Merge branch 'main' into add-flwr-log
danieljanes Sep 19, 2024
9c31cfa
Update args to align with run CLI
chongshenng Sep 19, 2024
c2e4c40
Reorganise args
chongshenng Sep 19, 2024
cd8abd9
Reorg args
chongshenng Sep 19, 2024
6ba765a
Merge branch 'main' into add-flwr-log
chongshenng Sep 19, 2024
f1a3950
Merge branch 'main' into add-flwr-log
chongshenng Sep 20, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions src/py/flwr/cli/app.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@

from .build import build
from .example import example
from .log import log
from .new import new
from .run import run

Expand All @@ -34,6 +35,7 @@
app.command()(example)
app.command()(run)
app.command()(build)
app.command()(log)

if __name__ == "__main__":
app()
79 changes: 79 additions & 0 deletions src/py/flwr/cli/log.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
# Copyright 2024 Flower Labs GmbH. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
"""Flower command line interface `log` command."""

import time
from logging import DEBUG, INFO

import grpc
import typer
from typing_extensions import Annotated

from flwr.common.grpc import GRPC_MAX_MESSAGE_LENGTH, create_channel
from flwr.common.logger import log as logger


def log(
run_id: Annotated[
int,
typer.Option(case_sensitive=False, help="The Flower run ID to query"),
],
period: Annotated[
int,
typer.Option(
case_sensitive=False,
help="Use this to set connection refresh time period (in seconds)",
),
] = 60,
chongshenng marked this conversation as resolved.
Show resolved Hide resolved
follow: Annotated[
bool,
typer.Option(case_sensitive=False, help="Use this flag to follow logstream"),
] = True,
) -> None:
"""Get logs from Flower run."""

def on_channel_state_change(channel_connectivity: str) -> None:
"""Log channel connectivity."""
logger(DEBUG, channel_connectivity)

# pylint: disable=unused-argument
def stream_logs(run_id: int, channel: grpc.Channel, duration: int) -> None:
chongshenng marked this conversation as resolved.
Show resolved Hide resolved
"""Stream logs from the beginning of a run with connection refresh."""

# pylint: disable=unused-argument
def print_logs(run_id: int, channel: grpc.Channel, timeout: int) -> None:
"""Print logs from the beginning of a run."""

channel = create_channel(
server_address="127.0.0.1:9093",
chongshenng marked this conversation as resolved.
Show resolved Hide resolved
insecure=True,
root_certificates=None,
max_message_length=GRPC_MAX_MESSAGE_LENGTH,
interceptors=None,
)
channel.subscribe(on_channel_state_change)

if follow:
try:
while True:
logger(INFO, "Streaming logs")
stream_logs(run_id, channel, period)
time.sleep(2)
chongshenng marked this conversation as resolved.
Show resolved Hide resolved
logger(INFO, "Reconnecting to logstream")
except KeyboardInterrupt:
logger(INFO, "Exiting logstream")
channel.close()
else:
print_logs(run_id, channel, timeout=1)