Server goes down immediately after the first client is fired up.

[vkullu]

This is my first post. If this forum is not appropriate for the issues I am facing let me know where I could ask?

Context: My objective is to evaluate FLARE for federated learning with application to omics data from healthcare systems (genomics, proteomics, medical records)

I am following the tutorial from: https://nvidia.github.io/NVFlare/quickstart.html#setting-up-the-application-environment-in-poc-mode on my mac.

I am able to start the server alright. However, as soon as I start the first trainer I get the following message

2022-01-05 11:25:06,239 - FederatedServer - INFO - starting insecure server at localhost:8002 deployed FL server trainer. Traceback (most recent call last): File "/Users/vkulkarni/opt/anaconda3/lib/python3.9/runpy.py", line 197, in _run_module_as_main return _run_code(code, main_globals, None, File "/Users/vkulkarni/opt/anaconda3/lib/python3.9/runpy.py", line 87, in _run_code exec(code, run_globals) File "/Users/vkulkarni/projects/nvidia/nvflare-env/lib/python3.9/site-packages/nvflare/private/fed/app/server/server_train.py", line 232, in <module> main() File "/Users/vkulkarni/projects/nvidia/nvflare-env/lib/python3.9/site-packages/nvflare/private/fed/app/server/server_train.py", line 101, in main admin_server = create_admin_server( File "/Users/vkulkarni/projects/nvidia/nvflare-env/lib/python3.9/site-packages/nvflare/private/fed/app/server/server_train.py", line 202, in create_admin_server admin_server = FedAdminServer( File "/Users/vkulkarni/projects/nvidia/nvflare-env/lib/python3.9/site-packages/nvflare/private/fed/server/admin.py", line 217, in __init__ AdminServer.__init__( File "/Users/vkulkarni/projects/nvidia/nvflare-env/lib/python3.9/site-packages/nvflare/fuel/hci/server/hci.py", line 120, in __init__ self.server_bind() File "/Users/vkulkarni/opt/anaconda3/lib/python3.9/socketserver.py", line 466, in server_bind self.socket.bind(self.server_address) OSError: [Errno 48] Address already in use 2022-01-05 11:25:35,897 - FederatedServer - INFO - starting insecure server at localhost:8002 deployed FL server trainer. 2022-01-05 11:25:35,904 - FedAdminServer - INFO - Starting Admin Server localhost on Port 8003 2022-01-05 11:25:35,904 - root - INFO - Server started (nvflare-env) vkulkarni@C02FJ2WYMD6N nvidia % Exception in thread Thread-1: Traceback (most recent call last): File "/Users/vkulkarni/opt/anaconda3/lib/python3.9/threading.py", line 973, in _bootstrap_inner self.run() File "/Users/vkulkarni/opt/anaconda3/lib/python3.9/threading.py", line 910, in run self._target(*self._args, **self._kwargs) File "/Users/vkulkarni/projects/nvidia/nvflare-env/lib/python3.9/site-packages/grpc/_server.py", line 883, in _serve if not _process_event_and_continue(state, event): File "/Users/vkulkarni/projects/nvidia/nvflare-env/lib/python3.9/site-packages/grpc/_server.py", line 847, in _process_event_and_continue rpc_state, rpc_future = _handle_call(event, state.generic_handlers, File "/Users/vkulkarni/projects/nvidia/nvflare-env/lib/python3.9/site-packages/grpc/_server.py", line 757, in _handle_call return _handle_with_method_handler(rpc_event, method_handler, File "/Users/vkulkarni/projects/nvidia/nvflare-env/lib/python3.9/site-packages/grpc/_server.py", line 733, in _handle_with_method_handler return state, _handle_unary_unary(rpc_event, state, File "/Users/vkulkarni/projects/nvidia/nvflare-env/lib/python3.9/site-packages/grpc/_server.py", line 637, in _handle_unary_unary return thread_pool.submit(_unary_response_in_pool, rpc_event, state, File "/Users/vkulkarni/opt/anaconda3/lib/python3.9/concurrent/futures/thread.py", line 163, in submit raise RuntimeError('cannot schedule new futures after ' RuntimeError: cannot schedule new futures after interpreter shutdown

Happy to hop onto Slack or other mechanisms you may have for realtime communication. Thank you.

[nvkevlu]

It looks like you are using Python 3.9, however at the current moment only Python 3.8 or Python 3.7 are supported. We are aware of issues with Python 3.9 and will be working to add support for it in the future.

[vkullu]

Thank you @nvkevlu that worked. Sorry for the oversight on my part. I can now proceed to the examples section.

[yhwen]

Hi Vinayak,
I noticed that you are using Python3.9 to run the NVFLare. Currently, we only support Python 3.7 and 3.8 right now. Please downgrade your Python version.

[vkullu]

Thank you @yhwen that worked. Sorry for the oversight on my part. I can now proceed to the examples section.